- Research article
- Open Access
Development and testing of indicators to measure coordination of clinical information and management across levels of care
BMC Health Services Research volume 15, Article number: 323 (2015)
Coordination across levels of care is becoming increasingly important due to rapid advances in technology, high specialisation and changes in the organization of healthcare services; to date, however, the development of indicators to evaluate coordination has been limited. The aim of this study is to develop and test a set of indicators to comprehensively evaluate clinical coordination across levels of care.
A systematic review of literature was conducted to identify indicators of clinical coordination across levels of care. These indicators were analysed to identify attributes of coordination and classified accordingly. They were then discussed within an expert team and adapted or newly developed, and their relevance, scientific soundness and feasibility were examined. The indicators were tested in three healthcare areas of the Catalan health system.
52 indicators were identified addressing 11 attributes of clinical coordination across levels of care. The final set consisted of 21 output indicators. Clinical information transfer is evaluated based on information flow (4) and the adequacy of shared information (3). Clinical management coordination indicators evaluate care coherence through diagnostic testing (2) and medication (1), provision of care at the most appropriate level (2), completion of diagnostic process (1), follow-up after hospital discharge (4) and accessibility across levels of care (4). The application of indicators showed differences in the degree of clinical coordination depending on the attribute and area.
A set of rigorous and scientifically sound measures of clinical coordination across levels of care were developed based on a literature review and discussion with experts. This set of indicators comprehensively address the different attributes of clinical coordination in main transitions across levels of care. It could be employed to identify areas in which health services can be improved, as well as to measure the effect of efforts to improve clinical coordination in healthcare organizations.
Healthcare systems are in a constant process of adaptation due to rapid advances in technology, new treatments, high specialisation and changes in the organization of health services . As a consequence, patients are seen by an ever-expanding array of different providers in a variety of locations, making coordination difficult [1, 2]. This is particularly challenging in the care of patients with chronic and multiple conditions, who tend to use healthcare services more frequently and use a greater array of services than other patients [3, 4]. Clinical coordination across levels of care should prevent wasteful duplication of diagnostic testing, perilous polypharmacy and conflicting care plans [5, 6]; thus the effects of clinical coordination extend beyond cost reduction through improving quality of care [7–9].
This study is set within a conceptual framework for analysing the performance of integrated healthcare networks, which is based on an extensive literature review [6, 10] and could be applied in any healthcare area that arranges to provide a coordinated continuum of services to a defined population. In this framework, clinical coordination, together with continuity of care and access to health services, is considered an intermediate objective of integrated healthcare networks and is regarded as a means by which to reach the ultimate objectives of quality of care, efficiency and equity of access [6, 10, 11]. To analyse the achievement of these objectives, both external and internal processes and contextual factors are taken into account, as well as the different perspectives (services, professionals and users) and approaches.
In this conceptual framework, clinical coordination is defined as the harmonious connection of the different health services needed to provide care to a patient throughout the care continuum in order to achieve a common objective without conflicts [10, 12]. Continuity of care refers to how individual patients experience coordination of services, and it is defined as the degree to which patients experience care over time as coherent and linked . Clinical coordination across levels of care consists of the coordination of both clinical information and clinical management [6, 10]. Clinical information coordination is the transfer and use of patients’ clinical information in order to harmonize activities between providers, and consists of two dimensions: transfer of clinical information and the use of this information . Clinical management coordination is the provision of care in a sequential and complementary way according to a healthcare plan shared by the different services and healthcare levels involved, and consists of three dimensions: care coherence (i.e., the existence of similar approaches and treatment objectives among professionals from different levels of care), follow-up across care levels (i.e., the adequate monitoring of the patient when there are transitions from one care setting to another) and accessibility across levels (provision of care without interruption across levels of care throughout the clinical episode of the patient) .
The results of clinical coordination can be assessed by analysing processes aimed at coordination or their outputs (immediate results of activities related to clinical coordination) or outcomes (final expected middle-long term results of clinical coordination, such as hospital readmissions or avoidable hospital admissions), and using different perspectives (services, professionals, users (continuity)). The focus of this study relies on measures to assess the outputs of clinical coordination across levels of care (primary and secondary) by using service-based indicators.
Despite the interest this subject has generated, there are still important gaps in terms of measures to assess clinical coordination across levels of care and the development of new indicators continues to be considered a priority in health policy and health services research [14, 15]. Many of the attempts to address this to date have focused on developing indicators to measure healthcare outcomes which are attributed to improvements in clinical coordination . However, the development of output indicators has been limited, and without this type of indicators it is not possible to conclude that outcomes in health care can be attributed to improvements in clinical coordination across levels of care .
Existing sets of indicators are usually designed to analyse a single dimension (e.g. transfer of information) or attribute (e.g. due completion of referral forms and discharge reports) of clinical coordination [17–20]. Those which address more than one dimension of clinical coordination are not exhaustive in their approach to clinical coordination and are often insufficiently operative or are not directed at the assessment of clinical coordination across levels of care [21–25]. Furthermore, the conceptual framework used to develop these measures is not generally explained in detail, so it is not obvious exactly which aspects of clinical coordination are being analysed or how measures relate to clinical coordination.
As a result of these issues, there is an overrepresentation of some dimensions of clinical coordination addressed by indicators, whilst other dimensions have scarcely been investigated . Studies have concentrated in particular on the transfer of clinical information [22–24, 27–30], especially in terms of completeness of information in discharge reports [22, 30–34] and to a lesser degree in emergency reports  and referral forms [20, 35], and on the follow-up of patients and accessibility across care levels [22, 24, 29, 30, 36]. Only a few studies have used indicators to measure clinical coherence between care levels [30, 37].
The aim of this study is to develop and test a set of output indicators to comprehensively evaluate clinical coordination across care levels of care, i.e. addressing both types of clinical coordination, information and management, and their dimensions and attributes.
The study consisted of two phases: in the first phase, a set of indicators to measure clinical coordination across levels of care was developed based on the literature review and expert discussions, and in the second phase, the set was tested in three different healthcare areas.
1. Development of a set of indicators to measure clinical coordination across levels of care
Identification of indicators: literature review
The study was based on the conceptual framework for analysing the performance of integrated healthcare networks [6, 10], which identifies two types of clinical coordination across levels of care (clinical information and clinical management) and five dimensions (transfer of information, use of information, care coherence, follow-up across levels and accessibility across levels). A systematic review of literature was undertaken to identify previously developed indicators. A computerised search of the following bibliographic databases was conducted: Pubmed, Social Science Citation Index, Science Citation Index, ECONLIT, CINAHL and LILACS, in addition to standard internet search engines such as Google. The search strategy included a combination of descriptors and keywords relating to clinical coordination (‘coordination of care’ or associated key terms with similar meaning), levels of care (‘primary care’, ‘secondary care’, ‘hospitalization’, ‘interface’, ‘cross-level’ or associated terms) and measurement tools (‘measure’, ‘indicator’ or associated key terms), making use of the Boolean operator ‘AND’. References from retrieved studies were also screened for possible omissions. The search was conducted in May 2011. Additional searches were conducted on the following organizations’ websites: Agency for Healthcare Research and Quality (AHRQ), World Health Organization (WHO), Pan American Health Organization (PAHO), Physician Consortium for Performance Improvement (PCPI), The Joint Commission, Agència de Qualitat i Avaluació Sanitàries de Catalunya (AQuAS), Observatori de Tendències de Serveis de Salut, the RAND corporation and the National Quality Forum (NQF). Studies in English, Spanish, Portuguese or Catalan which included one or more indicators of clinical coordination across levels of care were selected.
Selection and adaptation of indicators according to the different types and dimensions of clinical coordination
First, the indicators included in the selected studies were analyzed to identify which attributes of clinical coordination they addressed . They were subsequently grouped according to type, dimension and the attribute of clinical coordination across levels of care that they addressed (Fig. 1).
Second, three meetings took place with a team of 13 experts, who were either healthcare researchers with experience in the development of performance indicators and clinical coordination assessment or managers of healthcare services. Decisions concerning the selection and adaptation of indicators were made taking the scientific literature into account and when a consensus was reached among all participating members over three sessions of roundtable discussions. During the first meeting, it became obvious that in order to be applied, most of the indicators could not be generic but rather needed to be defined in relation to a specific disease. However, in order to gain a good grasp of the degree of coordination, a number of different diseases were included which require high levels of coordination across levels of care: diabetes mellitus type II, heart failure, chronic obstructive pulmonary disease (COPD) and breast, lung, bladder and colon cancer.
During the meetings, indicators were discussed and adapted based on the existing local clinical practice guidelines, which formed the basis on which standards of clinical coordination across levels of care were established [38–41] (for example, the guidelines allowed the team to determine when an urgent referral to secondary care is appropriate or to define the maximum acceptable time from discharge to a consultation in primary care). Each indicator was described in terms of numerator, denominator, target population, exclusion criteria, definition of terms involved, sources of data and bibliography .
For each indicator, the team discussed its relevance to clinical coordination across levels of care and its capacity to measure that for which it was designed (face validity). The team also discussed whether the indicator measured an aspect of care that was susceptible to being improved by services (opportunity for improvement), as well as the formulation of the indicator in such precise terms that it could be applied consistently within and between organizations, allowing for comparability (reliability). The experts identified the best sources of data to calculate each indicator (electronic medical record audit and clinical and administrative databases) and discussed its feasibility in terms of data availability and accuracy.
2. Test of the set of indicators
Design and settings
A retrospective cross-sectional study was conducted applying the set of indicators in three healthcare areas of the Catalan public healthcare system. The objectives were to evaluate the feasibility of the indicators (availability of valid, reliable and consistent data across the system) and to apply the indicators in three different healthcare areas in order to assess their usefulness in describing clinical coordination across levels of care.
Three healthcare areas were selected in order to represent the diversity of providers present in Catalonia: Baix Empordà (rural and semi-urban), the city of Girona (urban) and the Ciutat Vella district of Barcelona (urban), which all serve a population of 75,000-100,000. A single entity manages both primary and secondary care in Baix Empordà (Serveis de Salut Integrats Baix Empordà — SSIBE) and in Girona (Institut Català de la Salut — ICS). In Ciutat Vella, two public entities manage primary care (ICS and Institut de Prestacions d’Assistència Mèdica al Personal Municipal — PAMEM) and a different public entity manages secondary care (Parc Salut Mar). With regard to the coordination mechanisms used in these areas, patients served in Baix Empordà had a single electronic medical record for both care levels, whereas patients served in the other two areas had two shared but different electronic medical records for primary and secondary care. Several additional mechanisms have been implemented to improve clinical coordination across levels of care within the organizations, such as shared clinical guidelines, online consultations between primary care physicians and specialists, automated notification of primary care following hospital discharge and clinical case discussions between the two care levels.
Study population, data source and sample
The study population consisted of patients who had the selected conditions and who had used more than one care level, i.e. they were discharged from hospital, had received outpatient secondary care, were referred to secondary care or were newly diagnosed in primary care, depending on the indicator (Tables 1 and 2).
Two sources of data were used: a) electronic medical record audit, to calculate seven indicators (five related to clinical information coordination across levels of care and two related to clinical management coordination across levels of care); b) clinical and administrative electronic databases (which differ from patients’ individual electronic medical records in the fact that they collate patient data), to calculate twelve indicators (all related to clinical management coordination).
For indicators based on electronic medical record audits, the sample size was calculated to estimate proportions, which were expected to be around 0.50; the margin of error was ±0.15 and alpha error 0.05. The sample size required was 42 patients. A simple random sample without replacement was selected from records provided by primary care centres and hospitals. For indicators based on electronic databases, all records were selected.
Instructions for the data collection procedure were developed and systematically applied. Problems during data collection and analysis were recorded.
For indicators based on electronic medical record audits, data was retrieved by one researcher using standardized forms. For indicators based on databases, primary care centres and hospitals of the healthcare areas provided clinical and administrative electronic databases. Information was retrieved based on specified procedures, which had to be adapted to each information system.
In the case of dichotomous indicators, percentages were calculated, and 95 % confidence intervals were estimated when indicators were based on electronic medical record audits. Means and standard deviation were calculated for continuous indicators. Problems during data collection and analysis were discussed with the group in order to assess the applicability of the indicators and identify the main barriers to their implementation.
The principles of confidentiality and anonymity were upheld in the researchers’ conduct, reporting, and storage of data arising from this study, in accordance with European and Spanish legislation on ethical research . The study protocol was approved by the Ethical Committee for Clinical Research ‘Parc Salut Mar (2010/4124/I)’.
1. Development of a set of indicators to measure clinical coordination across levels of care
Identification of indicators: literature review
A total of 892 documents were identified: 863 from bibliographic databases, 11 from organizations’ websites and 18 from references in retrieved studies (Appendix 1). Of these documents, 862 were excluded because they did not describe nor use indicators of clinical coordination across levels of care, and 30 met the inclusion criteria, containing at least one indicator. From these documents, 52 indicators were initially identified [17, 19, 21–24, 27, 28, 30–34, 36, 37, 44–50].
Selection and adaptation of indicators according to the different types and dimensions of clinical coordination
The 52 indicators addressed 11 different attributes of clinical coordination across levels of care (Fig. 2): 3 related to clinical information coordination and 8 related to clinical management coordination. The dimension “use of information” was not addressed by any attribute or indicator. After two meetings, an initial set of 21 indicators was drawn up (Fig. 2), which addressed 10 of the 11 identified attributes, since it was not possible to establish an unambiguous criterion which would permit the identification of redundant consultations. The remaining attributes were represented by at least 1 indicator.
The final set of indicators was as follows (Table 1 and 2):
Clinical information coordination across levels of care: 7 indicators measure the transfer of clinical information across care levels, addressing the availability of inpatient and emergency discharge reports in primary care (four indicators) and the completeness of inpatient and emergency discharge reports and referral forms, including the transfer of information on new medication, medical tests, reasons for referral and information for patients (three indicators). No indicator addressed the dimension “use of transferred information” since it was not possible to identify or design any indicator measuring the effective use by professionals of information generated in the other care level.
Clinical management coordination across levels of care: 6 indicators address care coherence by measuring the coordinated management of medical testing in primary and secondary care of patients with heart failure and COPD (2 indicators), the adequacy of the referral of heart failure patients from primary care to non-urgent outpatient secondary care or emergency care (2 indicators) and the completion of the diagnostic process for heart failure, which requires coordination between the two care levels (1 indicator). Four indicators measure the follow-up of patients, addressing the communication between the hospital and the primary care centre when patients with heart failure and COPD are discharged (2 indicators) and their follow-up in primary care after being discharged (2 indicators). Finally, four indicators measure accessibility across care levels, specifically the time elapsed from the primary care referral of patients with heart failure or suspected cancer to their first specialist care appointment (2 indicators) and the time elapsed from the suspicion of cancer in primary care to cancer diagnosis or initiation of treatment (2 indicators).
2. Test of the set of indicators
Clinical care information: transfer of clinical information across levels of care
In the three healthcare areas, inpatient and emergency discharge reports were immediately available in primary care, since the two care levels share electronic medical records. In general, the quality of transferred clinical information was high (i.e. the clinical information required for the transfer of patients between care levels is duly registered; for example, in referral forms: background morbidity, current medical treatment and reason for referral), especially with respect to the completeness of inpatient and emergency discharge reports, although there are notable differences between areas (57.1 % of discharge reports duly completed in Baix Empordà as opposed to 95.2 % in Girona) (Table 3). In contrast, there were low percentages of duly completed referral reports in two of the healthcare areas (11.9 % and 26.2 % of reports).
All indicators were feasible in the three healthcare areas; however, in some cases the specified sample size was not reached due to an insufficient number of cases per year or due to insufficient precision in the available data to allow identification of the denominator – more than 100 records were reviewed without reaching the required sample size.
Clinical management coordination across levels of care
Indicators showed different degrees of test duplication (coordinated medical testing across levels of care) depending on the type of medical test performed: the highest level of duplication was observed in electrocardiograms for patients with heart failure (48 %) and the lowest was observed in spirometries for patients with COPD (2.5 %) (Table 4). In terms of care at the most appropriate care level, indicators showed high levels of adequate referral to non-urgent and emergency care. Finally, in two healthcare areas there were low percentages of patients (13.9 % and 22.7 %) who had had an echocardiogram performed in the year prior to the diagnosis of heart failure (completion of the diagnostic process).
Follow-up across care levels
In terms of communication, there were significant differences in the degree to which hospitals communicate with primary care prior to the discharge of heart failure or COPD patients (58 % and 3.2 % of COPD patient discharges in Baix Empordà and Ciutat Vella respectively). Similarly, with regard to follow-up after hospital discharge, there were marked differences between areas (follow-up of patients with COPD in primary care ranged from 26 % in Ciutat Vella to 76.7 % in Baix Empordà).
Accessibility across care levels
The average time waited to access secondary non-urgent care for heart failure patients referred to cardiology was higher than three weeks in all cases, with significant variations across areas. In contrast, the average time waited to access urgent care for patients with suspected cancer was lower than a week in all healthcare areas, with little variation between areas.
Five indicators were feasible in the three healthcare areas, six indicators were feasible in two healthcare areas, two indicators were feasible in only one area and lastly, one indicator was not feasible in any of the healthcare areas. Difficulties in calculating indicators were due to two types of problems. Firstly, problems related to the identification of the denominator: not possible to identify patients who had started insulin therapy (3 areas), patients referred for the first time to the secondary care level (2 areas), and patients referred to secondary care for suspected cancer (1 area). Secondly, problems related to the availability and accuracy of data needed to calculate the numerator: reason for seeking emergency care not recorded (2 areas), echocardiograms conducted in secondary care not systematically registered (1 area).
Clinical coordination is considered a health policy priority, as a lack of coordination can lead to poor quality of care and inefficiencies in the use of resources [5–9]. However, its measurement is still challenging [1, 14, 15], since calculating the degree of clinical coordination in its multidimensional nature requires the availability of indicators that cover the different types and dimensions of clinical coordination.
Until now, most attempts to tackle this challenge have focused on the design of indicators to measure certain outcomes which can potentially be attributed to clinical coordination . However, progress must be made in the design of instruments to measure the outputs of clinical coordination in order to be able to attribute improvements in the outcomes of health care to improvements in clinical coordination . With this in mind, this research constitutes a step forward by using a pre-established conceptual framework to generate a set of output indicators which address the two types of clinical coordination across levels of care (and most of their dimensions and attributes) that have been highlighted in previous studies . Furthermore, in contrast with previous efforts [17, 37], the indicators presented here have been described in operative terms, thus allowing for their precise application in healthcare organizations.
With regard to clinical information coordination across levels of care, seven indicators addressing the transfer of clinical information have been created. Applying these indicators has permitted the analysis of transfer of information in the three healthcare settings, taking in both evidence of information transfer between levels and the quality of the information transferred. No previous set of indicators has allowed researchers to address these two attributes jointly in main transitions between levels of care [1, 30, 34, 53], so this is one of its most significant contributions. In addition, the result of the applicability test has proven that these indicators have the accuracy and feasibility needed to make their calculation possible in different healthcare areas. Their joint application has revealed that, although there is a flow of information between the different care levels, the quality of information varies across transitions and organizations, thus leading to the identification of specific margins of improvement in each healthcare area.
It is important to highlight, however, that clinical information coordination is not fully represented by the set of indicators, since we were unable to address the use of transferred information; i.e. we could not determine whether information was actually read and used by the receiving professional . The lack of this type of measure of clinical coordination has been previously expressed in the literature [1, 54] and reflects the complexity of analyzing an activity which is not generally recorded but is considered central to clinical coordination. The fact that indicators are unable to systematically address all dimensions and attributes of clinical coordination points to the need to complement and enrich indicator results with those that can be obtained via different techniques, such as surveys or qualitative interviews with health professionals and patients.
With regard to clinical management coordination across levels of care, the systematic review led us to identify five attributes that define care coherence, two that define follow-up and one that defines accessibility across care levels. Their operationalization has resulted in a set of indicators to measure the main attributes of care coherence (such as coordinated medical testing across care levels or the provision of care at the most appropriate care level), follow-up (such as the existence of communication and follow-up after discharge) and accessibility across care levels (waiting time after referral). However, one of the eight identified attributes of care coherence, no redundant visits to primary and secondary care, is not represented by any indicator, since we were unable to establish an unambiguous criterion, either though the literature review or by expert consensus, which would permit the identification of redundant consultations.
During the first meeting, it became obvious that in order to be applied, most indicators could not be general but needed to be defined relating to a specific disease. However, in order to gain a good grasp of the degree of coordination in the area, a number of different diseases were included, which require high levels of coordination across levels of care: diabetes mellitus type II, heart failure, chronic obstructive pulmonary disease (COPD) and breast, lung, bladder and colon cancer.
The indicators to measure clinical management coordination have been adapted to several clinical conditions, due to the fact that the standards of clinical coordination upon which indicators are based need to be precise and based on what the evidence dictates, which varies according to the disease. Nevertheless, they can be adapted to other conditions as long as they have an evidence-based recommendation upon which to base the standard of clinical coordination measured by the indicator. Moreover, the use of the selected conditions (diabetes mellitus, chronic obstructive pulmonary disease, heart failure and cancer) could be considered a good strategy to identify the strengths and weakness in clinical coordination across levels of care [3, 55], since they meet the criteria to be considered adequate tracer conditions : care is provided across levels and over the course of time; the care that should be provided at each care level is well defined; they are among the most prevalent diseases in the population; diagnoses are well defined; and their epidemiology is well known.
The applicability test illustrates the usefulness of these indicators in describing clinical management coordination, pointing to areas for improvement, such as the coordination of medical testing in Baix Empordà and communication with primary care after discharge in Ciutat Vella. Furthermore, as they cover the main attributes of clinical management coordination across levels, they can be used to support the design of strategies to improve clinical coordination between levels of care.
It is important to note, however, that several problems arose which made the calculation of some of these indicators difficult or impossible in some healthcare areas. The problems were related to non-registration of the variables in information systems and under-registration of information by professionals, which points to the need for further improvements in information systems and record-keeping skills before we can systematically measure certain relevant aspects related to clinical management coordination across levels of care in these healthcare areas [56–59].
The methodology adopted in this study provides guarantees in terms of reliability of the indicators, since they have been adequately defined and precisely specified so that they can be implemented consistently within and across organizations (48). This is also true in terms of face and content validity, as the indicators have been adapted or newly created on the basis of scientific evidence and expert consensus. Furthermore, the applicability test provided information regarding data availability and accuracy (feasibility), thus highlighting major and minor problems in calculating the indicators, which could be informative for future studies. Finally, the indicators have been shown to be able to identify differences between areas, even in small samples. Further research should provide evidence regarding other relevant characteristics of the indicators, such as test-retest reliability and discriminant validity.
Certain aspects should be taken into account when applying the set of indicators in other healthcare contexts. First of all, data availability and validity should be explored. Secondly, it should be determined whether the information taken from the different health information systems is linkable, since indicators are constructed upon information generated in different levels of care. Lastly, although information recorded in digital format is desirable, most indicators could be calculated from a medical record audit, so data computerization is not a prerequisite.
One limitation of this study is the possibility of a publication bias in which relevant indicators were not identified (for example, indicators that measure clinical coordination but employ different terms, indicators published in other languages or grey literature not easily accessed by standard internet search engines). Moreover, the inclusion of terms referring to certain attributes of clinical coordination, such as “follow-up” or “referral adequacy”, might have extended the range of studies obtained. However, we employed several additional strategies for the identification of studies to reduce the possibility of publication bias, such as reviewing the reference lists of eligible documents and consulting the websites of the main organizations that design indicators. Another limitation is that one dimension (the use of clinical information) and one attribute (no redundant visits to primary and secondary care) are not represented by the set of indicators, pointing to the need to enrich the results obtained by the indicators with additional information from health professionals and patients in order to attain a more accurate evaluation of the process of coordination.
A set of rigorous and scientifically sound measures of clinical coordination were developed based on a literature review and discussion with experts. These indicators of clinical information and management coordination across levels of care could be employed to identify areas in which health care can be improved, as well as to measure the effect of efforts to improve clinical coordination. However, some relevant attributes of clinical coordination are not represented in the final set of indicators, which detracts from its comprehensiveness. In fact, clinical coordination is a multidisciplinary construct, and certain relevant dimensions and attributes of clinical coordination across levels of care such as the effective use of transferred information or redundant visits cannot be properly measured through indicators. Other approaches are therefore needed to obtain additional information, such as surveys or qualitative interviews. The indicators provided may also be useful for conducting comparative studies of clinical coordination across healthcare areas. Aspects such as the possibility of linking information from different health information systems, data availability and validity should be explored before proceeding to implement these indicators.
Reid R, Haggerty J, McKendry R. Defusing the confusion: concepts and measures of continuity of healthcare. Ottawa: Canadian Health Services Research Foundation; 2002.
Starfield BH. Atención primaria. [Primary Care: Balancing Health Needs, Services, and Technology]. Barcelona: Masson, S.A.; 2001.
Nolte E, McKee M. Caring for people with chronic conditions. A health system perspective. Maidenhead: European Observatory on Health Systems and Policies Series. Mc Graw Hill, Open University Press; 2008.
Vogeli C, Shields AE, Lee TA, Gibson TB, Marder WD, Weiss KB, et al. Multiple chronic conditions: prevalence, health consequences, and implications for quality, care management, and costs. J Gen Intern Med. 2007;22 Suppl 3:391–5.
Longest B, Young G. Coordination and communication. In: Shortell SM, Kaluzny AD, editors. Health care managment. Organization design and behavior. Albany, NY: Delmar; 2000. p. 210–43.
Vázquez ML, Vargas I, Unger JP, Mogollon AS, da Silva MRF, De Paepe P. Integrated health care networks in Latin America: toward a conceptual framework for analysis. Rev Panam Salud Publica. 2009;26:360–7.
Banks P. Policy framework for integrated care for older people. London: King's Fund; 2004.
Øvretveit J. Does clinical coordination improve quality and save money? Vol.1 A summary review of the evidence. London: Health Foundation; 2011.
Moore C, Wisnivesky J, Williams S, McGinn T. Medical errors related to discontinuity of care from an inpatient to an outpatient setting. J Gen Intern Med. 2003;18:646–51.
Terraza-Núñez R, Vargas I, Vázquez ML. Coordination among healthcare levels: systematization of tools and measures. Gac Sanit. 2006;20:485–95.
Vázquez ML, Vargas I, Unger JP, De Paepe P, Mogollón-Pérez AS, Albuquerque P, et al. Evaluating the effectiveness of care integration strategies in different health care systems in Latin America: the EQUITY-LA II quasi-experimental study protocol. BMJ Open 2015;5:E007037.
Shortell SM, Gillies RR, Anderson DA, Morgan KL, Mitchell JB. Remaking health care in America. San Francisco: The Jossey-Bass health care series; 1996.
Vargas I, Mogollón-Pérez AS, De Paepe P, da Silva MRF, Unger JP, Vázquez ML. Do existing mechanisms contribute to improvements in care coordination across levels of care in health services networks? Opinions of health personnel in Colombia and Brazil. BMC Health Serv Res 2015; 15:213.
U.S. Department of Health and Human Services. 2012 Annual Progress Report to Congress: National Strategy for Quality Improvement in Health Care. 2012. http://www.ahrq.gov/workingforquality/nqs/nqs2012annlrpt.pdf. Accessed 28 July2015.
Øvretveit J. Does clinical coordination improve quality and save money?Vol.2 A detailed review of the evidence. London: Health Foundation; 2011.
DuGoff EH, Dy S, Giovannetti ER, Leff B, Boyd CM. Setting standards at the forefront of delivery system reform: aligning care coordination quality measures for multiple chronic conditions. J Healthc Qual. 2013;35:58–69.
Bird JC, Beynon GJ, Prevost AT, Baguley DM. An analysis of referral patterns for dizziness in the primary care setting. Br J Gen Pract. 1998;48:1828–32.
Minvielle E, Leleu H, Capuano F, Grenier C, Loirat P, Degos L. Suitability of three indicators measuring the quality of coordination within hospitals. BMC Health Serv Res. 2010;10:93.
Sytema S, Micciolo R, Tansella M. Continuity of care for patients with schizophrenia and related disorders: a comparative south-Verona and Groningen case-register study. Psychol Med. 1997;27:1355–62.
Anderson MA, Helms LB. Extended care referral after hospital discharge. Res Nurs Health. 1998;21:385–94.
National Quality Forum. Quality Connections. Care Coordination. Washington, DC: National Quality Forum; 2010.
Wenger NS, Young RT. Quality indicators for continuity and coordination of care in vulnerable elders. J Am Geriatr Soc. 2007;55:S285–92.
Thomas RJ, King M, Lui K, Oldridge N, Pina IL, Spertus J. AACVPR/ACCF/AHA 2010 update: performance measures on cardiac rehabilitation for referral to cardiac rehabilitation/secondary prevention services: a report of the American Association of Cardiovascular and Pulmonary Rehabilitation and the American College of Cardiology Foundation/American Heart Association Task Force on Performance Measures (Writing Committee to Develop Clinical Performance Measures for Cardiac Rehabilitation). Circulation. 2010;122:1342–50.
Weinmann S, Roick C, Martin L, Willich S, Becker T. Development of a set of schizophrenia quality indicators for integrated care. Epidemiol Psichiatr Soc. 2010;19:52–62.
Ouwens M, Hermens R, Hulscher M, Vonk-Okhuijsen S, Tjan-Heijnen V, Termeer R, et al. Development of indicators for patient-centred cancer care. Support Care Cancer. 2010;18:121–30.
Schultz EM, Pineda N, Lonhart J, Davies SM, McDonald KM. A systematic review of the care coordination measurement landscape. BMC Health Serv Res. 2013;13:119.
Ouwens MM, Hermens RR, Termeer RA, Vonk-Okhuijsen SY, Tjan-Heijnen VC, Verhagen AF, et al. Quality of integrated care for patients with nonsmall cell lung cancer - Variations and determinants of care. Cancer. 2007;110:1782–90.
Thomas RJ, King M, Lui K, Oldridge N, Pina IL, Spertus J. AACVPR/ACC/AHA 2007 performance measures on cardiac rehabilitation for referral to and delivery of cardiac rehabilitation/secondary prevention services. Circulation. 2007;116:1611–42.
National Quality Forum. Preferred practices and performance measures for measuring and reporting care coordination. A consensus report. Washington, DC: National Quality Forum; 2010.
Foundation ABIM. American College of Physicians, Society of Hospital Medicine, The Physician Consortium for Performance Improvement. Care Transitions: Performance Measurement Set. Phase I: Inpatient Discharges & Emergency Department Discharges. American Medical Association: Chicago, IL; 2009.
Gandara E, Ungar J, Lee J, Chan-Macrae M, O'Malley T, Schnipper JL. Discharge documentation of patients discharged to subacute facilities: a three-year quality improvement process across an integrated health care system. Jt Comm J Qual Patient Saf. 2010;36:243–51.
Lee DS, Tran C, Flintoft V, Grant FC, Liu PP, Tu JV. CCORT/CCS quality indicators for congestive heart failure care. Can J Cardiol. 2003;19:357–64.
Tu JV, Khalid L, Donovan LR, Ko DT. Indicators of quality of care for patients with acute myocardial infarction. CMAJ. 2008;179:909–15.
van Walraven C, Taljaard M, Etchells E, Bell CM, Stiell IG, Zarnke K, et al. The independent association of provider and information continuity on outcomes after hospital discharge: implications for hospitalists. J Hosp Med. 2010;5:398–405.
Anderson MA, Hill PD. Psychometric properties of the referral data inventory. Appl Nurs Res. 1994;7:151–7.
Coleman EA, Eilertsen TB, Magid DJ, Conner DA, Beck A, Kramer AM. The association between care co-ordination and emergency department use in older managed care enrollees. Int J Integr Care. 2002;2, e03.
Wenger N, Young R. Quality Indicators of Continuity and Coordination of Care for Vulnerable Elder Persons. Santa Monica, CA: RAND; 2004.
Verdú JM, Pacheco V, Amado E, Esgueva N, López N, Alemany L, et al. Heart Failure. Barcelona: Institut Català de la Salut; 2008.
Anguera C, Caula J, Castillo J, Gaintano A, Llauger M, Pastor E, et al. Chronic obstructive pulmonary disease. Barcelona: Institut Català de la Salut; 2009.
Mata M, Cos F, Morros R, Diego L, Barrot J, Berengué M, et al. Diabetes mellitus II. Barcelona: Institut Català de la Salut; 2010.
Departament de Salut de la Generalitat de Catalunya. Rapid diagnostic circuit (CDR) of lung, colorectal, breast, bladder and prostate cancer; instructions 04/2005 and 02/2007. Barcelona: Generalitat de Catalunya; 2013.
Joint Commission. Attributes of core performance measures and associated evaluation criteria. 2010. http://www.jointcommission.org/Attributes_of_Core_Performance_Measures_and_Associated_Evaluation_Criteria/default.aspx. Accessed 28 July 2015.
Boletín Oficial del Estado. [Royal Decree 1720/2007 of 21th December, which approves the regulations for the development of the Organic Law 15/1999 of 13th December concerning the protection of personal data]. 2007.
Hermann RC, Palmer H, Leff S, Shwartz M, Provost S, Chan J, et al. Achieving consensus across diverse stakeholders on quality measures for mental healthcare. Med Care. 2004;42:1246–53.
American College of Physicians, Physician Consortium for Performance Improvement, National Committe for Quality Assurance. Emergency medicine: physician performance measurement set. Chicago, IL: American Medical Association; 2011.
American College of Physicians, Physician Consortium for Performance Improvement, National Committe for Quality Assurance. Melanoma II: physician performance measurement set. Chicago, IL: American Medical Association; 2010.
Hermens RP, Ouwens MM, Vonk-Okhuijsen SY, van der Wel Y, Tjan-Heijnen VC, van den Broek LD, et al. Development of quality indicators for diagnosis and treatment of patients with non-small cell lung cancer: a first step toward implementing a multidisciplinary, evidence-based guideline. Lung Cancer. 2006;54:117–24.
Sharma G, Fletcher KE, Zhang D, Kuo YF, Freeman JL, Goodwin JS. Continuity of outpatient and inpatient care by primary care physicians for hospitalized older adults. JAMA. 2009;301:1671–80.
van Walraven C, Taljaard M, Bell CM, Etchells E, Stiell IG, Zarnke K, et al. A prospective cohort study found that provider and information continuity was low after patient discharge from hospital. J Clin Epidemiol. 2010;63:1000–10.
Bertrán P. [Care coordination between levels. A proposal for evaluation]. Master thesis. Barcelona: Universitat Pompeu Fabra; 2006.
Boyd CM, Darer J, Boult C, Fried LP, Boult L, Wu AW. Clinical practice guidelines and quality of care for older patients with multiple comorbid diseases: implications for pay for performance. JAMA. 2005;294:716–24.
Aller MB, Vargas I, Waibel S, Coderch J, Sánchez-Pérez I, Llopart JR, et al. Factors associated to experienced continuity of care between primary and outpatient secondary care in the Catalan public healthcare system. Gac Sanit. 2013;27:207–13.
Ganz DA, Koretz BK, Bail JK, McCreath HE, Wenger NS, Roth CP, et al. Nurse practitioner comanagement for patients in an academic geriatric practice. Am J Manage Care. 2010;16:e343–55.
van Walraven C, Seth R, Austin PC, Laupacis A. Effect of discharge summary availability during post-discharge visits on hospital readmission. J Gen Intern Med. 2002;17:186–92.
Smith P, Mossialos E, Papanocloas I, Leatherman S. Performance measurement for health system improvement: experiences, challenges and prospects. Cambridge: Cambridge University Press; 2009.
McDonald KM, Schultz E, Chapman T, Davies S, Pineda N, Lonhart J, et al. Prospects for care coordination measurement using electronic data sources. Rockville: Agency for Healthcare Research and Quality; 2012.
Chung KP, Chang YJ, Lai MS, Kuo RN, Cheng SH, Chen LT, et al. Is quality of colorectal cancer care good enough? Core measures development and its application for comparing hospitals in Taiwan. BMC Health Serv Res. 2010;10:27.
Weiskopf NG, Weng C. Methods and dimensions of electronic health record data quality assessment: enabling reuse for clinical research. J Am Med Inform Assoc. 2013;20:144–51.
Terry AL, Chevendra V, Thind A, Stewart M, Marshall JN, Cejic S. Using your electronic medical record for research: a primer for avoiding pitfalls. Fam Pract. 2010;27:121–6.
Wenger NS, Roth CP, Shekelle PG, Young RT, Solomon DH, Kamberg CJ, et al. A practice-based intervention to improve primary care for falls, urinary incontinence, and dementia. J Am Geriatr Soc. 2009;57:547–55.
Ouwens MM, Hermens RR, Hulscher MM, Merkx MA, van den Hoogen FJ, Grol RP, et al. Impact of an integrated care program for patients with head and neck cancer on the quality of care. Head & Neck. 2009;31:902–10.
Direcció General de Planificació i Avaluació. [Strategy and Priorities in Cancer in Catalonia. Oncology Master Plan: objectives 2010]. Barcelona: Direcció General de Planificació i Avaluació; 2009.
The research leading to these results received funding from the Instituto de Salud Carlos III (PI10/00348) and Fondos FEDER. The funding source had no involvement in the study design, nor in the collection, analysis and interpretations of data, or in the writing of the article and the decision to submit it for publication.
The authors are most grateful to the people that participated in the study and generously gave their time and Claudia Ortiz and Josep Maria Lisbona who contributed to data collection. We thank Nuria Martinez for her administrative support and help, Irene Garcia Subirats for her statistical support, and Kate Bartlett for her help in correcting the English version of this article.
The authors declare that they have no competing interest.
MBA, IV and MLV were responsible for the study conception and design, for the systematic review and data analysis. MBA, IV, JC, SC, FC, MA, JF, JRL, LC and MLV participated in the selection, adaptation, development and interpretation of the indicators. JC, SC, FC, MA participated providing operational and methodological support to the fieldwork. MBA, IV and MLV drafted the manuscript. MBA, IV, JC, SC, FC, MA, JF, JRL, LC and MLV participated in data interpretation, reviewed draft versions of the paper and approved the final version. All authors read and approved the final manuscript.
Search Strategy and number of studies retrieved in the bibliographic databases
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
About this article
Cite this article
Aller, MB., Vargas, I., Coderch, J. et al. Development and testing of indicators to measure coordination of clinical information and management across levels of care. BMC Health Serv Res 15, 323 (2015). https://doi.org/10.1186/s12913-015-0968-z
- Quality indicators
- Coordination across levels of care
- Clinical management coordination
- Clinical information coordination
- Health services research