Skip to main content
Download PDF

Protocole of a controlled before-after evaluation of a national health information technology-based program to improve healthcare coordination and access to information

BMC Health Services Research volume 17, Article number: 297 (2017) Cite this article

Abstract

Background

Improvement of coordination of all health and social care actors in the patient pathways is an important issue in many countries. Health Information (HI) technology has been considered as a potentially effective answer to this issue. The French Health Ministry first funded the development of five TSN (“Territoire de Soins Numérique”/Digital health territories) projects, aiming at improving healthcare coordination and access to information for healthcare providers, patients and the population, and at improving healthcare professionals work organization. The French Health Ministry then launched a call for grant to fund one research project consisting in evaluating the TSN projects implementation and impact and in developing a model for HI technology evaluation.

Methods

EvaTSN is mainly based on a controlled before-after study design. Data collection covers three periods: before TSN program implementation, during early TSN program implementation and at late TSN program implementation, in the five TSN projects’ territories and in five comparison territories. Three populations will be considered: “TSN-targeted people” (healthcare system users and people having characteristics targeted by the TSN projects), “TSN patient users” (people included in TSN experimentations or using particular services) and “TSN professional users” (healthcare professionals involved in TSN projects). Several samples will be made in each population depending on the objective, axis and stage of the study.

Four types of data sources are considered: 1) extractions from the French National Heath Insurance Database (SNIIRAM) and the French Autonomy Personalized Allowance database, 2) Ad hoc surveys collecting information on knowledge of TSN projects, TSN program use, ease of use, satisfaction and understanding, TSN pathway experience and appropriateness of hospital admissions, 3) qualitative analyses using semi-directive interviews and focus groups and document analyses and 4) extractions of TSN implementation indicators from TSN program database.

Discussion

EvaTSN is a challenging French national project for the production of evidenced-based information on HI technologies impact and on the context and conditions of their effectiveness and efficiency. We will be able to support health care management in order to implement HI technologies. We will also be able to produce an evaluation toolkit for HI technology evaluation.

Trial registration

ClinicalTrials.gov ID: NCT02837406, 08/18/2016.

Peer Review reports

Background

The challenges of Health Information Technologies to improve healthcare coordination

All the health systems face challenges in delivering high-quality, effective and safe care at an affordable cost. Improvement of coordination of all health and social care actors in the patient pathways is an important issue in many countries [1–3]. Healthcare coordination is defined by the Agency for Healthcare Research and Quality as “the deliberate organization of patient care activities between two or more participants (including the patient) involved in a patient’s care to facilitate the appropriate delivery of health care services” [4]. Care coordination is thus the patient-centred organization of care that providers (including patient caregivers) should share to improve the quality of a patient’s management and, ultimately, the patient’s health [5–8]. The development of ambulatory care and the scattering of healthcare producers lead to increasing risk of care fragmentation [9]. Moreover, in the context of the increase in life expectancy and of the number of patients with chronic illnesses, continuity of care becomes a crucial condition for care quality and security [10, 11]. A key success factor to care coordination is sharing of the same holistic view of a patient’s condition by all actors of hospital, ambulatory and medico-social sectors, including the patient’s active diseases and current treatments, and the planned care pathway that establishes the role of each provider. Unfortunately, transitions from one setting or provider to another still frequently lack clear communication and coordination, resulting in an increase in healthcare costs and reduced quality of care including polypharmacy and adverse drug interactions, duplication of services, unnecessary emergency department utilization, high hospital readmissions rates, and in the worst cases patient injury [5, 9, 12, 13].

Given the large volume of transactions in the system and the need for new evidence-based practice and other information management activities, Health Information (HI) technology has been considered as a potentially effective answer to this issue [12, 14]. A recent WHO (World Health Organization) report shows that there is around Europe tangible progress in the mainstreaming of technology solutions to improve public health and health service delivery [15]. At the core of this technology-led transition is an adjustment in the way health information is captured, viewed, processed, exchanged and stored. HI technology involves a broad group of activities that use electronic means to deliver social and health-related information, resources and services.

HI technologies are potentially effective tools to improve information system between healthcare and social care professionals and between patients and health care professionals, allowing a better access to more precise information, improving communication and coordination between healthcare and social care organizations and healthcare professionals. They may allow the centralisation of all patient management information, the creation of shared tools for healthcare coordination, the creation of alerts, feedback, reminders thus favouring the implication of health and social care professionals and patients in their care pathway, continuous quality assessment and care efficiency as well as the development of a shared culture between professionals [15].

Literature on HI technology efficacy and efficiency shows controversial results. Even if many studies show a real but modest effect [16–20], others suggest the absence of positive effect, particularly on outcomes and cost-effectiveness indicators [17, 21–23], acknowledging a gap between the postulated and empirically demonstrated benefits of HI technologies [21, 24]. Some studies even indicate potential risks and unintended effects [25, 26].

These controversial results are explained by methodological reasons (global poor quality of the studies, heterogeneity of evaluated HI technologies and of HI technologies implementation's context, hampering the external validity of the studies) but also by pitfalls in HI technology development, often disregarding the interdependencies between technology, human characteristics and the socioeconomic environment [27–31]. Many researchers advocate larger, well-designed, controlled studies evaluating HI technology against a comprehensive set of measures, ideally throughout all stages of the technology’s life cycle [32, 33]. Such evaluation should be characterised by careful attention to socio-technical factors and organizational issues to maximise the likelihood of successful implementation and adoption [25, 26]. Another key issue for the development of HI technology is the political commitment, backed by sustainable funding. Published evidence of the information needed to make decisions about acquiring and implementing HI technology in different settings is nearly inexistent [15].

Healthcare coordination and HI technology development in France

In France, rules have been developed since the late 1990s to introduce healthcare system change for healthcare coordination improvement [34]. General Practitioners (GPs) have gained a major role in care coordination, thanks to a semi gate-keeping system that provides incentives to people to visit their GP prior to consulting a specialist. A coordinated care pathway was implemented with higher co-insurance for patients consuming care out of this pathway and new categories of co-payment for patients were created with the introduction of deductibles on some categories of care such as drugs, physicians and nurses consultations or patient transportation. Creation of primary healthcare centres that bring together professionals from different specialties is encouraged to improve coordination of care and cooperation between healthcare professionals. However, major problems include a lack of coordination between hospital and ambulatory services, between private and public provision of care and between healthcare and public health [34].

In this context, French public authorities are supporting HI technologies development. In 2014, a governmental grant, called TSN (“Territoire de Soins Numérique”/Digital health territories), selected five HI technology development projects across five pilot areas containing between 200,000 and 500,000 inhabitants each in five French regions: Aquitaine, Bourgogne, Ile-de-France, la Réunion and Rhônes-Alpes. These HI technology projects are supposed to introduce changes in healthcare organization, aiming at improving healthcare coordination and access to information for healthcare providers, patients or the population and at improving healthcare professionals’ work organization.

In September 2014, the French Health Ministry launched a call for grant to fund one project that should develop a method to evaluate the five TSN projects’ use and implementation and their impact on three main outcomes: healthcare quality, healthcare professionals’ work organization and efficiency. The selected project should also be able to create a general framework for the evaluation of HI technology services.

The EvaTSN project

This article presents the protocol of the project that was selected, called EvaTSN (Evaluation Territoire de Soins Numérique). It has been conceived by a multidisciplinary team composed of health services researchers, economists, sociologists, management researchers, HI technology researchers, epidemiologists and healthcare professionals. An operational team is in charge of the project lead and management.

The project’s main objectives consist in evaluating the TSN program implementation and impact and developing a model for HI technology evaluation. The specific objectives are structured around four axes: 1) TSN program implementation and use; 2) Impact of TSN program on patient pathway quality and safety, including access to care; 3) Impact of TSN program on healthcare professionals work organization; and 4) Economic sustainability and efficiency of the TSN program.

Methods

Design of the study

The study is mainly based on a controlled before-after study design (Fig. 1). Data collection covers periods before TSN program implementation (2012-2015 or T0 period), during early TSN program implementation (2016, T1 period) and at late TSN program implementation (starting in 2017, T2 period), in the five TSN projects territories and in five comparison territories. Analyses of TSN program implementation and use are based on a before-after design.

Fig. 1
figure 1

EvaTSN project data collection schedule and organization, France

Full size image

The interventions – the TSN projects

All the TSN projects develop digital services for healthcare users and healthcare and social care producers (hospitals, ambulatory sector, medico-social sector) to improve healthcare coordination, collaboration between professionals and access to healthcare and prevention information and professional guidelines (Table 1).

Table 1 Presentation of digital/E-services and functionalities in each TSN project area – EvaTSN project, France
Full size table

Most TSN projects target patients with chronic conditions or elderly dependent people. Some of them focus on patients suffering from specific diseases, including congestive heart failure (Aquitaine, Bourgogne, la RĂ©union regions), cancer (Bourgogne, Ile-de-France regions), stroke (Bourgogne), respiratory failure (Aquitaine), or specific populations, including young people (Bourgogne), pregnant women (Ile-de-France), obese and disabled people (la RĂ©union), and situations of social vulnerability (Bourgogne, Ile de France).

Study populations

Three populations will be considered: 1) “TSN targeted people”: healthcare system users and people having characteristics targeted by the TSN projects, 2) “TSN patient users” that is to say people included in TSN experimentations or using particular services and 3) “TSN professional users”, i.e. healthcare professionals involved in TSN projects. Several samples will be constituted in each population depending on the objective, axis and stage of the study.

Outcomes and measurements – data sources and data collection

Four types of data sources are considered (Table 2).

Table 2 EvaTSN project data collection: indicators collected, data collection tools and study populations, France
Full size table

Extractions from the French National Heath Insurance Database (SNIIRAM) will be conducted to collect the main set of indicators for axes 2, 3 and 4. This exhaustive nationwide database is at the heart of the financing system of diagnosis, drugs and physicians in ambulatory care settings and of independent practitioners in private hospitals (essentially fee for service). It provides data on claims paid for each patient by the Social Security System and is therefore the main source of information on ambulatory setting activity and associated expenditures. This database contains patient data (age, sex, place of living, long-term and chronic diseases, date of birth, date of death, health insurance scheme, benefit of free complementary insurance for lower-income people), all consultations and visits to GPs and ambulatory care specialists (but nothing about their content), all medical technical procedures, all dispensed medical devices and drugs, all lab and diagnostic tests but not their results, and providers’ level data (their activity and sales turnover, geography, prescribing behaviour). This database is also linked with the PMSI (programme de médicalisation du système d’information/Information system medicalization program) database providing more detailed information on hospital activity and expenditure, including diagnoses and procedures. Extractions from the SNIIRAM will cover the time period from 2012 to 2017 and will focus on samples from the “TSN targeted people” population, in both TSN projects and control areas. Data will also be extracted from the APA (Allocation personnalisée d’autonomie/personalized autonomy allowance) database. APA is a financial support for the elderly to cover home-care, nursing-care or institutional-care expenses.

Ad hoc surveys based on questionnaires will collect information on:

  • Knowledge of TSN projects assessed at T1 (early-implementation observation) and T2 (late-implementation observation) in a sample of the “TSN targeted people” population (axis 1);

  • TSN program use, ease of use, satisfaction and understanding assessed at T2 in a sample of the “TSN professional users” population (axis 3);

  • TSN use satisfaction and TSN pathway experience assessed at T2 in a sample of the “TSN patient users” population (axis 2);

  • Appropriateness of hospital admissions assessed using the validated French version of the Appropriateness Evaluation Protocol questionnaire (AEP) in most emergency services of the TSN and control areas, twice, before TSN program implementation (T0) and at early TSN program implementation (T1) (axis 2). The survey will be carried out by one clinical research assistant assessing the appropriateness of hospital admissions of the same week day for each TSN project area and its control area.

Qualitative analyses using semi-directive interviews and focus groups will be conducted in the five TSN project areas by a team of sociologists. Three samples will be included: a sample of around 50 TSN professional users interviewed at T0, T1 and T2, for the exploration of professionals’ coordination practices, their evolution and the level of TSN projects use and adoption by professionals (axis 3); based on a biographical approach (Illness narrative), a sample of 10 TSN projects patients-users interviewed monthly after TSN project implementation for the exploration of users’ participation to TSN projects and the analysis of the TSN program’s contribution to patient pathway (axis 2); and a sample of the TSN national program and the five TSN projects developers and main stakeholders interviewed at T1 to analyse the way projects have been implemented, what worked well and their barriers and difficulties. Patients, TSN professionals and TSN project developers will receive incentives for their participation to qualitative analyses. The sample size of 3 × 50 participants have been decided to favor heterogeneity of health-care professionals recruited. At each step (T1, T2 and T3), 50 health-care professionals will be recruited from the five TSN territories and from the five health- and social-care professional categories: general practitioner, private-sector specialist physician, private-sector nurse, physicians working at hospital, social care professional. Sample should also be heterogeneous according to TSN program implication.

A set of standardised TSN implementation indicators will be requested to TSN projects producers and extracted from each TSN project’s database.

Control territories

Each control territory is chosen to be as similar as possible to each corresponding TSN project territory, based on simple quantitative indicators. The identification method has already been used in France for the evaluation of another public policy (implementation of a minimal income for low-income people). Each control territory is chosen in the same region as the TSN project territory to control for regional policy effects. It must have roughly the same size as the TSN project territory, be a compact geographical zone and not be too geographically close to the TSN project territory, in order to avoid contamination. The selection process, described elsewhere, is carried in three steps:

  1. 1)

    Constitution of a list of candidate control territories, according to geographic, demographic and healthcare offer criteria;

  2. 2)

    Sorting of the list according to the territory’s structural similarity with the TSN project territory, based on a set of 20 criteria classified in four categories: 1) population demographic characteristics influencing healthcare demand; 2) population social and economic characteristics; 3) population health and 4) healthcare offer. The similarity analysis is based on factorial analyses;

  3. 3)

    Determining the territory that is the most similar to the TSN amongst the subset of candidate territories, according to the number of emergency room visits’ evolution, which is potentially influenced by TSN projects effects. This analysis is based on time series models. Were also considered in the final choice: feasibility of data collection and existence of competitive interventions in the territory.

Data analysis and interpretation

Analysis involves quantitative, qualitative and data visualisation approaches.

Quantitative analyses in axes 2, 3 and 4 mostly rely on a set of standardized indicators which represent the outcomes supposedly influenced by the TSN projects in quality and intensity of healthcare consumption or medical practices. These indicators are calculated on an annual or quarterly basis and compared over time (between T0, T1 and T2). Adjustment is done through direct standardization depending on the scope of the indicator: by age and sex for general population-based measures and, to the extent possible, by clinical and social background for indicators focused on particular populations (the elderly, patients witch chronic diseases…).

Statistical estimations are then performed to assess the impact of TSN projects outcomes, with inherent limitations. First, no overall influence of TSN projects can be produced, given the variety of services provided and the disparity from one region to another: estimations are thus restricted to particular services. Second, for both technical and methodological reasons, estimation cannot be performed on subsamples of TSN users, neither for patients nor for health professionals. Instead, estimates are calculated on the whole populations targeted by the services when relevant: elderly people with functional impairments for the coordination service, young adults for vaccination reminders, patients with chronic diseases (diabetes, heart failure…) for serious games or connected devices, etc.

This approach is usually referred to as “intention-to-treat” estimate (ITT), by contrast to local average estimates (LAE) calculated on program takers/users only. Causal effects of TSN services can then be assessed if and only if i) services are exclusively intended to specific subgroups of patients, ii) these groups can be routinely identified in SNIIRAM data and iii) there is a sufficient share of actual users of the services within these groups (otherwise, variance of ITT estimate is getting too large). For axis 1 analysis, indicators of TSN use, implementation and of TSN users characteristics will be described.

Qualitative analysis of axes 1, 2 and 3 is based on interviews which are recorded, anonymous, integrally transcribed and imported into the NVivo11 software to achieve a thematic content analysis. This method aims "at spotting, in verbal or textual expressions, recurring general themes that appear under various more concrete content" [35]. This type of analysis intends to "proceed systematically in the identification, consolidation and, alternatively, the discursive examination of themes in a body"[36]. The construction process of these thematic categories, coding, is both inductive and deductive because the development of themes and sub themes rests on both literature and emerging categories of empirical analysis. The exploratory qualitative studies require in-depth exploration of these emerging categories (Grounded Theory method).

Qualitative investigations point out political and organizational barriers to implementation (axis 1), the conditions of acceptance of HI Technology and organizational innovations, the TSN program’s effects on healthcare work and professional coordination practices (axis 3), the changes in care pathway if they are significant for the patient and their family caregivers (especially patient empowerment) (axis 2).

Monitoring and coordination

Audits of the conduct of the study are done every six months by the jury of the call, independently from the French Health Ministry. The EvaTSN project follow-up is also done yearly by an Independent Advisory Board.

Ethical considerations

The complexity of the evaluation implies separate approval for different facets. This study has been approved by the French National Institute of Health Data (IDS – Institut des données de santé). Approval by the national CCTIRS (Comité consultatif sur le traitement de l'information en matière de recherche dans le domaine de la santé) and the CNIL (Commission nationale Informatique et Libertés) has been obtained for SNIIRAM and for qualitative analyses.

Participants in qualitative analyses (semi-directive interviews) are guaranteed strict confidentiality of records and of all statements through an encrypted identifier for each participant. Each participant fills in an informed consent form before the beginning of the study, given to the sociologist in charge of the qualitative analyses. The files and audiorecordings are kept in a sealed location of the research centre, with access limited to the qualitative analysis researchers of the EvaTSN research group, and will be destroyed after 15 years. No name will appear on any public documents and no information will be divulged that could allow participants to be identified by a third party. The EvaTSN research group members have access to anonymous quantitative datasets. All the quantitative analyses and quantitative data collection from questionnaire or databases are strictly anonymous.

Results will be disseminated as a report to the French Health Ministry. An disseminating plan to participants is in discussion with the French Health Ministry. Authorship of articles is defined according to the Vancouver Convention.

Discussion

We expect the project to add information on HI technology effectiveness, efficiency, and effectiveness determinants in the French healthcare system. It should produce key elements for HI technology effective development and for structuring further evaluation of HI technology impact.

Many authors point out that there is a lack of evidence-based HI technology which is much needed [21, 24, 32]. Results from these studies could pave the way to a strong evidence-based implementation strategy of digital technologies in a local system, aiming at providing patient-centred and quality-assured coordinated care.

Moreover, many authors acknowledge the lack of care coordination in developed countries and the need to build and test interventions to improve care coordination [37]. Some trials testing the impact of HI technology intervention on care coordination are ongoing or already carried out in American and European countries, focusing on several conditions (depression, cardiovascular disease, cognitive impairment) [38–40]. But many HI technology interventions still need to be designed and tested in a variety of settings. Care coordination is the next opportunity and challenge for HI technology. Recently, a call to methodologically robust research on HI technology impact on care coordination was launched by the International Medical Informatics Association (IMIA) [24]. Our research project is the first one conducted in France which covers such a large perimeter of types of care and disorders or patient conditions.

Since complex interventions like HI technology include multiple components that are interrelated or interdependent, it can be challenging to develop, document, evaluate and report on them [41]. Our project will apply the principles of complex intervention evaluation, combining quantitative and qualitative analyses, conducting impact and process evaluation with multiple evaluation criteria collected from different sources (“TSN targeted people”, “TSN patient users”, “TSN professional users”, TSN developers and stakeholders, French National Heath Insurance Database). Process evaluation is essential for different reasons. It allows the understanding of the right contexts for HI technologies and their components to serve as an essential support to the delivery of patient-centred, coordinated, and quality-assured care. It also allows the evaluation of organizational changes that are paramount for the impact of new HI technology tools.

Impact evaluation using different dimensions is also needed. We evaluate the effect of TSN projects on final and intermediate health outcomes (for example: prevalence of drug overuse or misuse, prevalence of hospital admissions’ appropriateness, incidence of chronic diseases, etc.) and on patient satisfaction and experience with HI supports and coordination [42].

TSN projects are developed in five territories with rather similar size (about 200 000 inhabitants) and social profiles (high proportion of low-income population). They are geographically spread in five regions on the whole French territory and have both rural and urban areas. They represent the geographical diversity of the French territory. This may favour a good external validity of the study.

This complex evaluation has many methodological challenges.

We have to develop a common evaluation framework for five TSN projects with several HI technology components targeting different populations. This implies to have a good knowledge of all the TSN projects components, to identify the common core components of TSN projects, to have a good understanding of their action mechanisms and to rely on many dimensions (structure, process, and outcome) covering the whole spectrum of potential TSN projects’ effects. This diversity may improve mainly the external validity of our conclusions thus allowing the identification of the most effective HI technology interventions.

Another methodological challenge is linked to the gap between the evaluation and the TSN project intervention level. Each TSN project is focused on specific conditions or diseases (specific chronic conditions, social vulnerability situations, elderly dependant people, etc.) and includes samples of patients and people presenting these conditions (“TSN patient users”). Our main quantitative analyses use data from the French National Heath Insurance Database (SNIIRAM database). As it is impossible to identify the TSN patient users individually speaking, we need to select groups of people presenting similar characteristics as those targeted in the TSN projects and living in the TSN territory (“TSN targeted people”) using validated algorithms. This lack of specificity in TSN projects evaluation may lower the strength of the association between TSN projects interventions and outcomes. However the estimates correspond to the real impact assessment of TSN project at a moderate development stage. Only qualitative analyses can precisely and specifically identify and interview TSN projects users (“TSN patient users” or “TSN professional users”), in complementary means of quantitative analyses.

The third methodological challenge is linked to the controlled before-after study design. We make the strong hypothesis of the comparability of each TSN project territory with its control at the beginning and throughout all the TSN projects implementation and evaluation. We use a precise and proven method to identify the control TSN project territories, based on structural and social similarity. However, a strong hypothesis is that the control and the TSN project territories are subject to the same external factors that may influence evaluation criteria.

We face other difficulties. First, the French Health Ministry has given a short schedule. First conclusions should be given in 2017 and the evaluation should be finished in 2018. It is a very short time as the TSN projects are still being implemented and moreover, parts of some projects are in ongoing development. It is a real difficulty for the production of conclusions about TSN program health impact because we will not be able to see all the TSN projects' effects yet at that time. The HI technology evaluation model will be however described. Second, the TSN projects are still being developed although the evaluation has already started. This situation allows time for the before evaluation and leads to more contacts with TSN projects developers; it is an advantage for the precise understanding of the TSN projects components and action mechanisms. However, it is delaying the outcome and process evaluation criteria definition. Finally, the consortium has to be very careful, when dealing with interlocutors at the Ministry to keep full independence regarding strategic and methodological issues.

EvaTSN is a challenging French national research project for the production of evidenced-based information on HI technologies impact and on the context and conditions of their effectiveness and efficiency. We will be able to support health care management and policy decision making in order to implement HI technologies.

We will also be able to produce an evaluation toolkit for HI technology evaluation. This toolkit will consist in adequate study design and schedule for evaluation, appropriate evaluation criteria, data collection sources and means. It will be based on our EvaTSN large HI technology program evaluation experience.

Abbreviations

AEP:

Appropriateness evaluation protocol

APA:

Allocation personnalisée d’autonomie

CCTIRS:

Comité Consultatif sur le Traitement de l’Information en matière de Recherche dans le domaine de la Santé

CNIL:

Commission Nationale Informatique et Libertés

EvaTSN:

Evaluation Territoire de Soins Numérique

GPs:

General practitioners

HI technology:

Health information technology

IDS:

Institut des Données de Santé

IMIA:

International Medical Informatics Association

ITT:

Intention to treat

LAE:

Local average estimate

PMSI:

Programme de Médicalisation du Système d’Information

SNIIRAM:

French National Health Insurance Database

TSN:

Territoires de Soins Numérique

WHO:

World Health Organization

References

  1. Osborn R, Moulds D, Squires D, Doty MM, Anderson C. International survey of older adults finds shortcomings in access, coordination, and patient-centered care. Health Aff. 2014;33:2247–55.

    Article  Google Scholar 

  2. Suter E, Oelke ND, Adair CE, Armitage GD. Ten key principles for successful health systems integration. Healthc Q Tor Ont. 2009;13(Spec No):16.

    Article  Google Scholar 

  3. Dykes PC, Samal L, Donahue M, Greenberg JO, Hurley AC, Hasan O, et al. A patient-centered longitudinal care plan: vision versus reality. J Am Med Inform Assoc. 2014;21:1082–90.

    Article  PubMed  PubMed Central  Google Scholar 

  4. McDonald KM, Sundaram V, Bravata DM, Lewis R, Lin N, Kraft SA, et al. Closing the Quality Gap: A Critical Analysis of Quality Improvement Strategies (Vol. 7: Care Coordination) [Internet]. Rockville (MD): Agency for Healthcare Research and Quality (US); 2007 [cited 21 Jun 2016]. (AHRQ Technical Reviews). Available from: http://www.ncbi.nlm.nih.gov/books/NBK44015/

  5. Bodenheimer T. Coordinating care--a perilous journey through the health care system. N Engl J Med. 2008;358:1064–71.

    Article  CAS  PubMed  Google Scholar 

  6. Little P, Everitt H, Williamson I, Warner G, Moore M, Gould C, et al. Observational study of effect of patient centredness and positive approach on outcomes of general practice consultations. BMJ. 2001;323:908–11.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  7. Davis K, Schoenbaum SC, Audet A-M. A 2020 vision of patient-centered primary care. J Gen Intern Med. 2005;20:953–7.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Mead N, Bower P. Patient-centred consultations and outcomes in primary care: a review of the literature. Patient Educ Couns. 2002;48:51–61.

    Article  PubMed  Google Scholar 

  9. SĂ©roussi B, Jaulent M-C, Lehmann CU. Health information technology challenges to support patient-centered care coordination. Yearb Med Inform. 2015;10:8.

    Article  PubMed  PubMed Central  Google Scholar 

  10. Kripalani S, Jackson AT, Schnipper JL, Coleman EA. Promoting effective transitions of care at hospital discharge: a review of key issues for hospitalists. J Hosp Med. 2007;2:314–23.

    Article  PubMed  Google Scholar 

  11. Kripalani S, LeFevre F, Phillips CO, Williams MV, Basaviah P, Baker DW. Deficits in communication and information transfer between hospital-based and primary care physicians: implications for patient safety and continuity of care. JAMA. 2007;297:831–41.

    Article  CAS  PubMed  Google Scholar 

  12. Haggerty JL, Reid RJ, Freeman GK, Starfield BH, Adair CE, McKendry R. Continuity of care: a multidisciplinary review. BMJ. 2003;327:1219–21.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Brown JB, Lewis L, Ellis K, Stewart M, Freeman TR, Kasperski MJ. Mechanisms for communicating within primary health care teams. Can Fam Physician. 2009;55:1216–22.

    PubMed  PubMed Central  Google Scholar 

  14. Liyanage H, Correa A, Liaw S-T, Kuziemsky C, Terry AL, de Lusignan S. Does informatics enable or inhibit the delivery of patient-centred, coordinated, and quality-assured care: a delphi study: a contribution of the IMIA Primary Health Care Informatics Working Group. IMIA Yearb. 2015;10:22–9.

    Article  CAS  Google Scholar 

  15. World Health Organization. Regional Bureau for Europe. From innovation to implementation: eHealth in the WHO European region. Copenhagen: World Health Organization. 2016.

  16. Brundisini F, Giacomini M, DeJean D, Vanstone M, Winsor S, Smith A, et al. Chronic disease patients’ experiences with accessing health care in rural and remote areas: a systematic review and qualitative meta-synthesis. Ont Health Technol Assess Ser. 2013;13:1–33.

    CAS  Google Scholar 

  17. Elbert NJ, van Os-Medendorp H, van Renselaar W, Ekeland AG, Hakkaart-van Roijen L, Raat H, et al. Effectiveness and cost-effectiveness of ehealth interventions in somatic diseases: a systematic review of systematic reviews and meta-analyses. J Med Internet Res. 2014;16(4):e110.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Holstiege J, Mathes T, Pieper D. Effects of computer-aided clinical decision support systems in improving antibiotic prescribing by primary care providers: a systematic review. J Am Med Inform Assoc [Internet]. 2014 [cited 21 Jun 2016]; Available from: http://jamia.oxfordjournals.org/cgi/doi/10.1136/amiajnl-2014-002886

  19. Collins S, Hurley AC, Chang FY, Illa AR, Benoit A, Laperle S, et al. Content and functional specifications for a standards-based multidisciplinary rounding tool to maintain continuity across acute and critical care. J Am Med Inform Assoc. 2014;21:438–47.

    Article  PubMed  Google Scholar 

  20. Gurwitz JH, Field TS, Ogarek J, Tjia J, Cutrona SL, Harrold LR, et al. An electronic health record-based intervention to increase follow-up office visits and decrease rehospitalization in older adults. J Am Geriatr Soc. 2014;62:865–71.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Black AD, Car J, Pagliari C, Anandan C, Cresswell K, Bokun T, et al. The Impact of eHealth on the Quality and Safety of Health Care: A Systematic Overview. Djulbegovic B, editor. PLoS Med. 2011;8(1):e1000387.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Chaudhry B, Wang J, Wu S, Maglione M, Mojica W, Roth E, et al. Systematic review: impact of health information technology on quality, efficiency, and costs of medical care. Ann Intern Med. 2006;144:742–52.

    Article  PubMed  Google Scholar 

  23. Ekeland AG, Bowes A, Flottorp S. Effectiveness of telemedicine: a systematic review of reviews. Int J Med Inf. 2010;79:736–71.

    Article  Google Scholar 

  24. Seroussi B, Jaulent MC, Lehmann CU. Looking for the Evidence: value of health informatics editorial. Yearb Med Inform. 2013;8:4–6.

    CAS  PubMed  Google Scholar 

  25. Steichen O, Gregg W. Health information technology coordination to support patient-centered care coordination. IMIA Yearb. 2015;10:34–7.

    Article  CAS  Google Scholar 

  26. Melby L, Hellesø R. Introducing electronic messaging in Norwegian healthcare: unintended consequences for interprofessional collaboration. Int J Med Inf. 2014;83:343–53.

    Article  Google Scholar 

  27. Graetz I, Reed M, Shortell SM, Rundall TG, Bellows J, Hsu J. The association between EHRs and care coordination varies by team cohesion. Health Serv Res. 2014;49:438–52.

    Article  PubMed  Google Scholar 

  28. Kern LM, Edwards A, Kaushal R. The patient-centered medical home, electronic health records, and quality of care. Ann Intern Med. 2014;160:741–9.

    Article  PubMed  Google Scholar 

  29. Carrillo JE, Carrillo VA, Guimento R, Mucaria J, Leiman J. The NewYork-presbyterian regional health collaborative: a three-year progress report. Health Aff. 2014;33:1985–92.

    Article  Google Scholar 

  30. Lund S, Rasch V, Hemed M, Boas IM, Said A, Said K, et al. Mobile phone intervention reduces perinatal mortality in Zanzibar: secondary outcomes of a cluster randomized controlled trial. JMIR MHealth UHealth. 2014;2(1):e15.

    Article  PubMed  PubMed Central  Google Scholar 

  31. van Gemert-Pijnen JEWC, Nijland N, van Limburg M, Ossebaard HC, Kelders SM, Eysenbach G, et al. A holistic framework to improve the uptake and impact of eHealth technologies. J Med Internet Res. 2011;13(4):e111.

    Article  PubMed  PubMed Central  Google Scholar 

  32. Demiris G, Kneale L. Informatics systems and tools to facilitate patient-centered care coordination. IMIA Yearb. 2015;10(1):15–21.

    Article  CAS  Google Scholar 

  33. Bouamrane M-M, Mair FS. Implementation of an integrated preoperative care pathway and regional electronic clinical portal for preoperative assessment. BMC Med Inform Decis Mak. 2014;14:93.

    Article  PubMed  PubMed Central  Google Scholar 

  34. Chevreul K, Durand-Zaleski I, Barahmi S, Hernandez-Quevedo C, Mladovski P. France. Health System Review. European Observatory on Health systems and Policies; 2010 291. (Health System in Transition). Report No.: 12.

  35. Saldana J. The Coding Manual for Qualitative Researchers. London: Sage; 2009.

    Google Scholar 

  36. Paille P, Mucchielli A. L’analyse qualitative en sciences humaines et sociales. Paris: Armand Colin; 2008.

    Google Scholar 

  37. Bates DW. Health information technology and care coordination: the next big opportunity for informatics? IMIA Yearb. 2015;10:11–4.

    Article  CAS  Google Scholar 

  38. Dalal AK, Roy CL, Poon EG, Williams DH, Nolido N, Yoon C, et al. Impact of an automated email notification system for results of tests pending at discharge: a cluster-randomized controlled trial. J Am Med Inform Assoc. 2014;21:473–80.

    Article  PubMed  Google Scholar 

  39. Walters P, Barley EA, Mann A, Phillips R, Tylee A. Depression in primary care patients with coronary heart disease: baseline findings from the UPBEAT UK study. PloS One. 2014;9(6):e98342.

    Article  PubMed  PubMed Central  Google Scholar 

  40. Ngandu T, Lehtisalo J, Levälahti E, Laatikainen T, Lindström J, Peltonen M, et al. Recruitment and baseline characteristics of participants in the Finnish Geriatric Intervention Study to Prevent Cognitive Impairment and Disability (FINGER)-a randomized controlled lifestyle trial. Int J Environ Res Public Health. 2014;11:9345–60.

    Article  PubMed  PubMed Central  Google Scholar 

  41. Craig P, Petticrew M. Developing and evaluating complex interventions: reflections on the 2008 MRC guidance. Int J Nurs Stud. 2013;50:585–7.

    Article  PubMed  Google Scholar 

  42. Scholl I, Zill JM, Härter M, Dirmaier J. An integrative model of patient-centeredness - a systematic review and concept analysis. PloS One. 2014;9(9):e107828.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgement

We thank all actors of the EvaTSN Research group (supplementary file) and TSN projects for their support.

The Eva TSN Research Group:

General coordination: Louis-Rachid Salmi (Univ. Bordeaux, ISPED, Centre INSERM U1219 Bordeaux population health research center, Bordeaux, France: principal investigator), Emmanuelle Fourneyron (Univ. Bordeaux: project manager), Sylvie Calvar (Univ. Bordeaux: administrative assistant)

Group 1: evaluation of use and deployment effectiveness

Louis-Rachid Salmi (Univ. Bordeaux, ISPED, Centre INSERM U1219 Bordeaux population health research center, Bordeaux, France: group 1 scientific co-coordinator), Frantz Thiessard (Univ. Bordeaux, INSERM U1219 Bordeaux population health research center, Equipe de recherche en informatique appliquée à la santé: group 1 scientific co-coordinator), Stéphane Pincemail (Univ. Bordeaux: group 1 operational coordinator), Viviane Ramel (Univ. Bordeaux, INSERM U1219 Bordeaux population health research center, Economie et management des organisations de santé), Jean-Philippe Joseph (Univ. Bordeaux, département de médecine générale), Sylvie Duhamel (Univ. Bordeaux, département de médecine générale), Tamara Roberts (Univ. Bordeaux), Maelys Abraham (Univ. Bordeaux)

Group 2: evaluation of efficacy and impacts on patients

Florence Saillour-Glenisson (Univ. Bordeaux, CHU de Bordeaux, INSERM U1219 Bordeaux population health research center, Economie et management des organisations de santé: group 2 scientific co-coordinator), Laetitia Huiart (CHU de La Réunion: group 2 scientific co-coordinator), Thomas Renaud (Univ. Bordeaux, INSERM U1219 Bordeaux population health research center, Economie et management des organisations de santé: group 2 operational coordinator), Tamara Roberts (Univ. Bordeaux), Mathieu Castry (Univ. Bordeaux), Elodie Kerdavid (Univ. Bordeaux)

Group 3: evaluation of efficacy and impacts on health professionals

Emmanuel Langlois (Univ. Bordeaux/Sciences Po Bordeaux, Centre Emile Durkheim: Group 3 scientific co-coordinator), Matthieu Sibé (Univ. Bordeaux, INSERM U1219 Bordeaux population health research center, Economie et management des organisations de santé: group 3 scientific co-coordinator), Sandrine Cueille (Univ. Pau et Pays de l’Adour: Group 3 scientific co-coordinator), Tamara Roberts (Univ. Bordeaux: group 3 operational coordinator), Thomas Renaud (Univ. Bordeaux, INSERM U1219 Bordeaux population health research center, Economie et management des organisations de santé), Nora Arditi (Univ. Bordeaux), Maelys Abraham (Univ. Bordeaux: litterature review), Jean-Philippe Joseph (Univ. Bordeaux, département de médecine générale), Sylvie Duhamel (Univ. Bordeaux, département de médecine générale)

Group 4: evaluation of economic efficiency

Jérôme Wittwer (Univ. Bordeaux, INSERM U1219 Bordeaux population health research center, Economie et management des organisations de santé: group 4 scientific coordinator), Thomas Renaud (Univ. Bordeaux, INSERM U1219 Bordeaux population health research center, Economie et management des organisations de santé: group 4 operational coordinator), Mathieu Castry (Univ. Bordeaux), Camille Jean (ENSAM Paris Tech, Laboratoire Conception de Produits et Innovation)

Control territories definition group:

Sophie Buffeteau, Yannick L’Horty, André Ochoa (Fédération nationale des observatoires régionaux de santé), Nadège Thomas (Fédération nationale des observatoires régionaux de santé)

Clinical Research Department of Bordeaux University Hospital:

Joaquin Martinez (Head of Department) – 12, rue Dubernat – 33400 Talence (France)

Advisory Board

Marc Cuggia (Univ. Rennes, CHU de Rennes), Serge Briançon (Univ. Nancy, CHU de Nancy, Ecole de Santé Publique de Nancy), Laure Com-Ruelle (IRDES, Paris), Nathalie Pelletier-Fleury (Centre de recherche médecine, sciences, santé, santé mentale, société, Paris), Claude Sicotte (Univ. Montréal, Groupe de recherche interdisciplinaire en santé, Montréal, Québec, Canada)

Funding

This study was financially supported by a grant from Ministry of Health (France) – PREPS Program, 2014, n°PREPS14001103N.

Availability of data and material

Not applicable.

Authors’ contributions

Design of the study: all authors. Writing the article: all authors. Comment and editing of review drafts: all authors. All authors read and approved the final manuscript.

Competing interests

The authors declared that they have no competing interests.

Consent for publication

Not applicable.

Ethic approval and consent to participate

EvaTSN project fulfilled the requirements for PREPS research projects. According to French regulatory on PREPS, approval of an ethical review board was not necessary for this kind of evaluation.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Author information

Authors and Affiliations

  1. ISPED, Centre INSERM U1219 Bordeaux Population Health Research Center, University Bordeaux, 146 rue LĂ©o Saignat, 33076, F-33000, Bordeaux, France

    Florence Saillour-Glénisson, Viviane Ramel, Thomas Renaud, Matthieu Sibé, Frantz Thiessard, Jerome Wittwer & Louis Rachid Salmi

  2. INSERM, ISPED, Centre INSERM U1219 Bordeaux Population Health Research Center, 146 rue LĂ©o Saignat, 33076, F-33000, Bordeaux, France

    Florence Saillour-Glénisson, Viviane Ramel, Thomas Renaud, Matthieu Sibé, Frantz Thiessard, Jerome Wittwer & Louis Rachid Salmi

  3. CHU de Bordeaux, Pôle de Santé Publique, Service d’Information Médicale, 146 rue Léo Saignat, 33076, F-33000, Bordeaux, France

    Florence Saillour-Glénisson & Louis Rachid Salmi

  4. Département de Médecine Générale, Université de Bordeaux, 146, rue Léo Saignat 33076, Bordeaux, Cedex, France

    Sylvie Duhamel, Jean Philippe Joseph & Tamara Roberts

  5. Institut de Santé Publique d’Epidémiologie et de Développement, Université de Bordeaux, 146, rue Léo Saignat 33076, Cedex F-33000, Bordeaux, France

    Emmanuelle Fourneyron & Stephane Pincemail

  6. CHU, Unité de Soutien Méthodologique, Saint-Denis, F-97400, France

    Laetitia Huiart

  7. University Bordeaux/Sciences Po Bordeaux, Centre Emile Durkheim, F-33000, Bordeaux, France

    Emmanuel Langlois & Tamara Roberts

  8. Groupe Hospitalier Est Réunion, Route Nationale 3 30, 97470, Saint-Benoît-de-Carmaux, France

    Laetitia Huiart

  9. Centre Emile Durkheim, Science Politique et Sociologie Comparatives, Université de Bordeaux, 3Ter, Place de la Victoire 33076, Bordeaux, Cedex, France

    Emmanuel Langlois

Authors
  1. Florence Saillour-Glénisson
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sylvie Duhamel
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Emmanuelle Fourneyron
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Laetitia Huiart
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jean Philippe Joseph
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Emmanuel Langlois
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Stephane Pincemail
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Viviane Ramel
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Thomas Renaud
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Tamara Roberts
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Matthieu Sibé
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Frantz Thiessard
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Jerome Wittwer
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Louis Rachid Salmi
    View author publications

    You can also search for this author in PubMed Google Scholar

Consortia

for the EvaTSN Research group

Corresponding author

Correspondence to Louis Rachid Salmi.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Saillour-Glénisson, F., Duhamel, S., Fourneyron, E. et al. Protocole of a controlled before-after evaluation of a national health information technology-based program to improve healthcare coordination and access to information. BMC Health Serv Res 17, 297 (2017). https://doi.org/10.1186/s12913-017-2199-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12913-017-2199-y

Keywords

BMC Health Services Research

ISSN: 1472-6963

Contact us