Case management and self-management support for frequent users with chronic disease in primary care: a pragmatic randomized controlled trial
© Chouinard et al; licensee BioMed Central Ltd. 2013
Received: 30 November 2012
Accepted: 15 January 2013
Published: 7 February 2013
Chronic diseases represent a major challenge for health care and social services. A number of people with chronic diseases require more services due to characteristics that increase their vulnerability. Given the burden of increasingly vulnerable patients on primary care, a pragmatic intervention in four Family Medicine Groups (primary care practices in Quebec, Canada) has been proposed for individuals with chronic diseases (diabetes, cardiovascular diseases, respiratory diseases, musculoskeletal diseases and/or chronic pain) who are frequent users of hospital services. The intervention combines case management by a nurse with group support meetings encouraging self-management based on the Stanford Chronic Disease Self-Management Program. The goals of this study are to: (1) analyze the implementation of the intervention in the participating practices in order to determine how the various contexts have influenced the implementation and the observed effects; (2) evaluate the proximal (self-efficacy, self-management, health habits, activation and psychological distress) and intermediate (empowerment, quality of life and health care use) effects of the intervention on patients; (3) conduct an economic analysis of the efficiency and cost-effectiveness of the intervention.
The analysis of the implementation will be conducted using realistic evaluation and participatory approaches within four categories of stakeholders (Family Medicine Group and health centre management, Family Medicine Group practitioners, patients and their families, health centre or community partners). The data will be obtained through individual and group interviews, project documentation reviews and by documenting the intervention. Evaluation of the effects on patients will be based on a pragmatic randomized before-after experimental design with a delayed intervention control group (six months). Economic analysis will include cost-effectiveness and cost-benefit analysis.
The integration of a case management intervention delivered by nurses and self-management group support into primary care practices has the potential to positively impact patient empowerment and quality of life and hopefully reduce the burden on health care. Decision-makers, managers and health care professionals will be aware of the factors to consider in promoting the implementation of this intervention into other primary care practices in the region and elsewhere.
KeywordsChronic diseases Primary care Family Medicine Group Frequent users Case management Self-management Primary care nursing Services integration
Because of their high prevalence and significant effects, chronic diseases (CD) pose a major challenge for health care and social services, for society and for the persons affected . These individuals often have to make important day-to-day adjustments as a result of disability, loss of income, and a declining quality of life [2, 3]. It is now recognized that primary care should be central to the customized and effective management of CD [4, 5], and that innovative strategies targeting the current organization of primary care must be offered and evaluated to better support the individuals affected , particularly the most vulnerable . To meet the complex needs of people with CD and to reduce societal consequences, primary care must provide a range of services that are interdisciplinary, person-centred and adjusted to the individual’s current health conditions and characteristics [7, 8]. These services must also be oriented towards self-management, where the people affected and their families are called upon to play a greater role in the management of their health [1, 7, 9].
A number people with CD require higher intensity care because of personal characteristics that increase their vulnerability. This applies especially to the socioeconomically disadvantaged [10, 11] and to those who present a comorbid mental health condition [12, 13] or multimorbidity (two or more CD) [13, 14]. In addition to a compromised quality of life and an increased risk of social isolation, these individuals have problems complying with treatment, adopting healthy behaviours and managing their health. This can result in increased services use, like emergency department visits and hospitalizations [15, 16]. Case management strategies could be developed to address the vulnerability factors of frequent users in order to prevent inequities in health care and related costs .
In 2004, Family Medicine Groups (FMG) were implemented by the ministère de la santé et des services sociaux du Québec (Quebec’s ministry of health and social services) to improve accessibility, continuity, and coordination of health care in Quebec . A FMG is an administrative arrangement for existing practices in which primary care physicians who wish to participate are grouped together to collaborate with nurses to offer primary care services, including patient follow-up, health promotion and preventive care, to a group of registered patients. It offers access to care 10 hours a day, seven days a week, through regular appointments, walk-in clinics, home visits, and after-hours health coverage using telephone hot-lines and emergency on-call services. Family physicians who are members of FMG will also work closely with other health care professionals in community services centres, hospitals, community pharmacies, etc., to complement the services they offer . Since the creation of these new care teams, FMG nurses have already improved health education and the accessibility and continuity of services for certain patients, including those with diabetes, hypertension, on anticoagulant treatment, etc. . However, the most vulnerable groups still pose major challenges in terms of accessibility, delivery and coordination of primary care . The current work organization in FMG cannot optimally respond to the multiple requests, the considerable needs for self-management support of these patient groups, and their frequent need to access various health care resources , due, among other things, to lack of coordination and integration of services.
A major consultation process conducted in 2010 on the organization of CD services in the Saguenay-Lac-Saint-Jean (SLSJ) region of Québec, identified two potential solutions to meet the challenges posed by vulnerable patients with CD who frequently use hospital services : (1) Improve the coordination of services through case management; and (2) Develop strategies to support self-management. Some vulnerable patients could benefit from closer monitoring by a case management nurse [1, 23] within a primary care team linked to other network resources [1, 6], and from self-management support [2, 6]. Case management programs for frequent users of emergency departments have already been developed or are being developed in the six Centre de santé et de services sociaux (CSSS) (health and social services centres) of the SLSJ region, including the two CSSS participating in the project. Without being formally evaluated, an assessment of their programs has brought to light several positive points : a significant decrease in the number of frequent users of emergency services; a high level of satisfaction among users and stakeholders; and close cooperation between the program coordinator and FMG nurses, who are considered essential partners for patients with CD. Although promising, these case management programs are often limited to the most serious cases due to capacity issues (50 persons per year at the CSSS de Chicoutimi).
Faced with the growing needs and primary care challenges posed by increasingly vulnerable patients, the Agence de santé et de services sociaux of the SLSJ region (regional health and social services agency) and the two partner CSSS proposed to implement similar and complementary primary care interventions to allow vulnerable patients with CD to benefit from case management by a nurse within their FMG. The expansion of case management within FMG will allow, together with the case management services already offered by the CSSS, a better response to the complex needs of vulnerable patients, as well as improved services integration. Case management will be performed in the primary care setting, the FMG, ensuring a better collaboration between case management nurse and family doctor . The presence of a primary contact (FMG nurse), who is a generalist and accessible, will promote the coordination of patient care. As a result, the provision of case management can be adjusted in intensity and duration based on patient needs, ensuring continuity and long-term management.
The proposed intervention seeks to address many of the challenges posed by CD, based on scientific evidence. First, case management by primary care nurses has proven to be effective for various CD [25–27]. In fact, a key element of the effectiveness of an interdisciplinary approach to CD for vulnerable primary care patients is the use of a single caregiver (usually a nurse) to serve as the main contact and to coordinate interventions between health care professionals and the services provided . However, most studies on this issue have been conducted in the context of a specific CD, which does not correspond to the reality of the clientele currently managed in primary care. Moreover, the implementation of strategies for interdisciplinary patient follow-up in the management of CD has to be evaluated in its context.
To date, strategies to support self-management remain poorly implemented in FMG. However, the positive effects of self-management support groups, such as the Stanford Chronic Disease Self-Management Program, are widely recognized [29, 30]. These strategies are intended mostly for people who are in an early stage of their disease . A recent study in primary care conducted in Ontario (Canada) showed a reduction in hospital length of stay and an increase in patient satisfaction with this type of self-management program . Another study, in which vulnerable CD patients in primary care were evaluated after a six-week intervention to support self-management, showed positive effects with regard to patient self-management capacity, but there was no comparison with a control group . Very few studies have examined the implementation mechanisms and the involvement of primary care professionals in such self-management support programs . The introduction of a self-management group support program in FMG may inpsire more vulnerable people to participate and attain positive outcomes .
This project aims to document the implementation and effects, within four FMG of the SLSJ region (Quebec, Canada), of a pragmatic intervention involving case management by a nurse to promote interdisciplinary person-centred follow-up and group self-management support for frequent users of hospital services (emergency department visits and hospitalizations) with CD (diabetes, cardiovascular diseases, respiratory diseases, musculoskeletal diseases and/or chronic pain). The evaluation of the intervention has three objectives: (1) to analyze the implementation of the intervention within the existing structures of the four participating FMG in order to: (a) Explain how the various contexts have influenced the implementation of the intervention and the observed effects, and (b) Identify elements that can be assessed and applied in order to improve the intervention and to promote its implementation in other FMG; (2) to evaluate the proximal (self-efficacy, self-management practices, health habits, activation and psychological distress) and intermediate (empowerment, quality of life and health care use) effects of this intervention among patients; (3) Conduct an economic analysis of the cost-effectiveness and cost-benefit of the intervention.
The theoretical framework of the intervention is based on two conceptual models. One supports the methodology of the clinical intervention, and the other supports the implementation process, change management, and knowledge transfer.
The first model is that of the UK National Health Service on innovation in health care and social services for people with CD . This model incorporates the basic principles of the Chronic Care Model , while also drawing on lessons learned from US models, such as that of Kaiser Permanente, with regard to the intensity of care that is appropriate for the complex needs of patients , and of the Evercare model, with regard to the use of case management nurses in primary care . The goal is to improve the health and quality of life of people with CD by providing personalized and ongoing support, based on the best evidence in the field. This model proposes the implementation of a case management system for patients with complex needs by making primary care a central part of the organization of services. To achieve this, the model suggests a structured approach that will allow for interaction between CSSS partners and community resources in order to provide integrated services. It also proposes the implementation of self-management support practices.
The second model the Promoting Action on Research Implementation in Health Services (PARiHS) [38, 39]. According to this model, successful implementation depends on the nature and type of evidence from previous studies, the results of the proposed study, the context in which it is introduced, and how the process is facilitated. The value of the evidence depends not only on its scientific reliability, but also on the experience of the professionals and partners, as well as on patient preferences. The implementation of evidence into practice is achieved through a dialogue with knowledge users and must take their views into consideration. Some settings are more amenable to implementation than others, particularly where there are natural leaders. Finally, this model emphasizes the importance of appropriate facilitation, including various strategies for managing change and increasing the chances of a successful implementation.
Case management by FMG nurses
The first component of the intervention is the follow-up offered under the case management process, which is seen as a collaborative, dynamic and systematic approach to ensure and coordinate care and services for a defined clientele based on interdisciplinary practice. Here the nurse evaluates, plans, implements, coordinates and prioritizes options and services according to patient health needs, in close collaboration with the involved partners . The intervention focuses on four main components: (1) A thorough evaluation of patient needs and resources; (2) Establishment and maintenance of a patient-centred, individualized service plan (ISP); (3) Coordination of services among partners; and (4) Self-management support for patients and families .
The main duties and responsibilities of nurses in case management will be to: (1) Evaluate the patient’s situation and needs and involve the family, with the patient’s consent; (2) Identify which partners of the CSSS and of the community network to involve; (3) Jointly plan patient follow-up by establishing an ISP with the partners and with the active participation of the patients and family — minimum of two ISP per patient: one to initiate the intervention and the other approximately 3 months later; (4) Negotiate the services and defend the rights and interests of the individual; (5) Coordinate care and services; (6) Monitor the ISP application; (7) Educate and support the person. The ISP formulation stage will be oriented towards a self-management support approach that will emphasize the following: the patient’s potential; setting objectives according to his or her perspective; developing problem-solving skills; and using the patient’s usual support system . Home visits by the nurse may be required, when justified — as in the case of reduced mobility or if a home assessment is needed — while avoiding overlap with home support services already offered.
The intervention will be implemented in each FMG by two nurses. Given the expertise of the nurses already working in the FMG and their established relationships with the medical teams and partners (e.g., community pharmacists), it is preferable that they implement the intervention rather than nurses hired for this project. This also promotes the sustainability of the intervention in the participating FMG. Nurses selected to conduct the intervention will receive five days of theory and practical training specific to case management and self-management support, provided by the Clinical Project Coordinator (described in following paragraph). Family doctors will be actively sought throughout the intervention to: identify participating patients, problems and intervention priorities, participate in ISP preparation meetings (in person or by having shared their perspective with the nurse prior to the meeting), and provide the medical components of ISP implementation. Partners include the CSSS professionals identified based on the needs of participants, such as psychosocial service providers (e.g. social worker) currently involved in a patient’s care, or additional professional resources specializing in CD management (nurse, nutritionist, etc.). The partners also include community organizations (e.g., home care services), patient associations and community pharmacists.
A Clinical Project Coordinator with experience in case management will be hired to coordinate and facilitate the implementation of the intervention (including change management). She will be responsible for: (1) setting up the intervention (training of case management nurses, development of materials and practice tools, recruitment and support of lay (peer) leaders for the self-management group support program, organization of support groups, scoping of organizations and community associations in the region and establishing the first contact to ensure their cooperation, etc.), including the communication plan to prepare participating FMG and all partners, in order to facilitate change management; and (2) ensuring the implementation and proper roll-out of the intervention (maintaining an appropriate standardized level of intervention and providing support for case management nurses, etc.). The Clinical Project Coordinator will be responsible for clinical governance in coordinating care within the network of each CSSS . This clinical governance will allow the Clinical Project Coordinator to establish the required intervention treatment pathways, with the support of the two primary care managers involved, as needed.
Self-management group support program
The second component of the intervention consists of group meetings (10–12 participants) for self-management support in accordance with the Stanford program . Developed by the Stanford Patient Education Centre (http://patienteducation.stanford.edu) to support people living with chronic conditions, the standardized curriculum, materials and program implementation, have made it the most accessible program for clinical and research applications . An estimated 30% of participants enrolled in the project should agree to participate in this component, based on past primary care experiences described in the literature [32, 33]. The self-management group support program rests on the premise that all patients with CD have similar concerns. They have the capacity to take responsibility for managing several aspects of their CD and will experience better results by building their sense of self-efficacy and developing the required skills. This model proposes a standardized six-week program with interactive weekly group meetings led by two lay leaders, who themselves have a CD. Twenty lay leaders (not participating in the intervention) will be recruited from the participating FMG practices and patient associations through a variety of media. Interviews will be conducted to ensure these volunteers are interested and able to provide this program. Two certified master trainers from My Toolbox program (http://mytoolbox.mcgill.ca) at McGill University will train lay leaders according to the Standard four-day training program. These lay leaders will conduct a complete simulation exercise with volunteer patients from their FMG to familiarize themselves with, and standardize, the process before conducting meetings with study participants. Two sessions of the self-management group support program will be implemented in each FMG. These meetings will take place at different times, depending on the availability of participants.
The evaluation of the intervention is based on a mixed design of complex health intervention evaluations . It focuses on three combined strategies based on the targeted objectives and a pragmatic vision of the evaluation. These three strategies: the implementation analysis, the evaluation of the effects and the economic analysis will be described separately. The methodological decisions regarding patient selection, the implementation of the intervention and the intervention modalities were made in accordance with the established FMG practices to ensure the intervention meets their needs, and take into account the organization of services.
Parameters of the implementation evaluation
Description of practice settings (contextual factors) (C)
Description of the current processes, patient integration and satisfaction (C)
Issues related to the implementation (C)
Evolution of the processes and integration
Identification of problems and failures (C)
Fidelity of the intervention (I)
Opinion about the implementation process (C)
Identification of obstacles and facilitating elements (C)
Description of the impact on stakeholders/organizations (E)
Satisfaction with the intervention (E)
Data collection strategies
A multiple-case study  will be performed with multiple sources of information. Such a strategy is particularly appropriate for analyzing the context and the mechanisms involved [50, 51]. Each FMG will be considered a case with its own context, and will include its interactions with partners. This study will be conducted with a multifaceted approach before, during and after the intervention. As described in Table 1, four categories of key informants will be interviewed: (1) FMG stakeholders (doctors/nurses); (2) managers (FMG/CSSS); (3) participating patients and their families; (4) identified partners of case management (CSSS professionals, representatives of community organizations and community pharmacists), as well as volunteer leaders of self-management groups. Four strategies for collecting data will be used: (1) focus groups; (2) individual interviews; (3) review of documents produced during the implementation of the intervention, (4) an intervention fidelity checklist. Interview guides with open-ended questions that are adapted to each category of key informants will be developed and validated by implementation evaluation experts.
The pre-implementation phase will describe the context of the intervention regarding: (1) the characteristics of the environment (contextual factors); (2) the operation and integration of existing services, as well as participant satisfaction; (3) the issues related to the implementation as identified through group discussions. Focus groups will involve FMG doctors (n = 6, one group per FMG), patients and their families (n = 4 patients, one group per FMG; n = 4 family members, one group per FMG) and partners considered to be key participants (n = 4). Individual interviews will be conducted with FMG and CSSS managers (n = 6) and FMG nurses (n = 8).
The implementation phase will identify the changes in FMG processes (mechanisms) and the integration of services mid-way through the implementation in addition to learning about the obstacles and challenges encountered, as reported through focus groups with FMG stakeholders (n = 6) and obtained from individual interviews with FMG and CSSS managers (n = 6). Documentation from meetings of the advisory committee and from meetings between the Clinical Project Coordinator and nurses or managers, etc., will be obtained and analyzed in order to identify obstacles and challenges, and adjustments made along the way. An evaluation of the fidelity of the intervention will be conducted to document the degree to which the intervention was implemented; this will vary with the needs of participating patients . After each encounter, nurses will conduct the evaluation for participating patients using a standardized checklist that includes the case management criteria and follow-up parameters, the details of ISP, the number of interdisciplinary meetings, the partners called upon and their involvement, and the number of self-management support meetings. These reports will help determine whether the interventions have conformed to the original study model.
The post-implementation phase will describe the implementation process, the obstacles and enabling factors, the effects of the intervention on stakeholders/organizations, and the satisfaction of key players. This will be accomplished through focus groups with FMG physicians (n = 6), patients and their families (n = 8) and partners considered key informants (n = 4), and individual interviews with FMG and CSSS managers (n = 6) and with FMG nurses (n = 8).
Data analysis plan
Data collected from the key stakeholders will be analyzed in three steps according to a qualitative content analysis procedure to identify emerging themes and trends: coding, sorting of documentation by content, and analysis. Driven by the data, inferences will be drawn and information units will be compared . This content analysis will be performed using NVivo software (Version 9) for data from individual and group interviews. In addition to seeking to reveal the themes that are specific to each element of the implementation evaluation (context, intervention, mechanisms of action, impact), the analysis at this stage aims to: (1) explain how different contexts have influenced the implementation of the intervention and the observed effects; and (2) identify elements to be considered in order to improve the intervention and promote its implementation in the other FMG . The scientific rigour of this approach will be assured by recognized qualitative research strategies . Credibility (accuracy of the description of the phenomenon) will be assured by open-ended questions, allowing participants some latitude in what they want to reveal and by the triangulation of informants. Reliability (accuracy with which we account for various perceptions expressed by participants) will be assured by data coding by at least two members of the research team with expertise in different domains (triangulation of researchers). Validation (confirmation of the analysis and interpretation) will be assured through extensive documentation of the analysis process. Transferability (transfer of the outcomes to other settings or populations) will be assured by the detailed description of the context and participants.
Evaluation of the effects
The evaluation of the effects on patients will be based on a pragmatic randomized experimental design with delayed intervention for the control group and measurements taken before and after the intervention (at six-month follow-up). This allows to evaluate interventions in actual clinical settings to maximize their generalizability . Implementation analysis will document how different contexts influenced the implementation and the observed effects. The before/after experimental design is a robust method for observing the potential effects of the intervention . The design also includes an assessment regarding use of services over a year in the experimental group.
Targeted patients will be those described in the objectives. The concept of vulnerability will be operationalized by considering both the frequency of the CSSS care use (emergency room visits and hospitalization) and patient vulnerability characteristics, based on the judgment of the primary care team in the FMG. This combined approach was proposed as a more favourable strategy for patient identification for this kind of intervention, when compared to using each one of these strategies in isolation . Steps to target the most vulnerable participants will include: 1) Presentation of the intervention to FMG primary care physicians and nurses, 2) Delivery of a computerized list (obtained using MAGIC Chronique software by MédiaMed Technologies ) of the 300 most frequent users of CSSS services (emergency department visits and hospitalizations) having at least one of the targeted CD in each FMG. FMG professionals will then identify the 100 patients they believe will benefit most from the intervention described. The software used for patient identification will serve as a system to support decision-making for primary care teams who did not have access to this kind of information up till now.
After verifying patient eligibility and obtaining their consent, participants will be allocated to one of two groups (experimental or control) according to a three-stage randomization process : (1) generation of the patient allocation sequence using a simple randomization procedure; (2) allocation blinding (sequentially numbered and sealed in opaque envelopes); (3) assignment of participants by opening the envelopes at the time of recruitment. The experimental group will consist of a sample of 50 patients in each of the four FMG (n = 200). These patients will receive the intervention for six months. Patients in the control group (n = 200) will receive the usual care for six months and then the same intervention as the treatment group for the following six months (waiting list control group) . This is intended to avoid the problem of demoralization of the control group, in addition to ensuring fair treatment among participants in both groups.
The variables defined within the logic model will be measured using instruments that are well known and have been validated in studies similar to the one proposed.
Proposed study plan
Time from the start of the intervention
Verification of eligibility
Covariables: Socioeconomic level, social isolation, literacy, mental health and multimorbidity
Proximal outcomes: Personal self-efficacy, self-management practices, lifestyles, activation and psychological distress
Intermediate outcomes: Empowerment, quality of life and use of services
Intermediate outcomes, evaluated at T0 and T2 (see Table 2), will include: (1) empowerment, measured with the heiQ scale ; (2) quality of life, measured with the SF-12v2 scale ; and (3) the use of health services, measured using clinical data from the CSSS (with Magic Chronique software by MediaMed) for hospitalizations, emergency room visits and CSSS services (e.g., psychosocial services or specialized services related to the specific CD), and electronic data on visits with physicians and nurses in the FMG.
A number of covariables will also be documented (T0) to describe participant characteristics: (1) socioeconomic status with family income and patient perceptions of his or her economic situation; (2) social isolation, measured with the social isolation subscale of the Nottingham Health Profile; (3) literacy, measured with the Newest Vital Sign; (4) mental health, measured with the Hospital Anxiety and Depression Scale (HADS) [70, 71]; and (5) multimorbidity, measured with the Disease Burden Morbidity Assessment (DBMA) . French versions are available for all validated instruments. When selecting data collection instruments, valid shorter versions were preferred to reduce completion time. Considering the circumstances of the vulnerable clientele, questionnaires will be self-administered in the presence of a research assistant who will provide the required assistance to the participant, ranging from minimal supervision to reading all the questions, if and when needed. If the patient cannot travel, the questionnaires can be administered by telephone or at the participant’s home.
The evaluation component will be conducted according to the study plan described in Table 2.
Sample size and statistical power
When analyzing quantitative data, we must take into account that part of the effect is potentially associated with each of the eight case management nurses. The intra-nurse or intracluster correlation (ρ) is unknown, however, and is specific to each variable. On the other hand, given that series of standardization measures will be implemented to minimize nurse-specific effects, we anticipate that the intracluster correlation will not be higher than 0.10. Sample size was therefore calculated using this maximal anticipated ρ-value. Results indicated that a standardized effect size (ES) of 0.5 (qualified as medium ) will be detectable with two groups of 200 participants (25 per nurse) for a two-tailed test with the α level set at 0.05 and a power of 80% . An ES of at least this magnitude can be expected since it was observed in a previous study  and in unpublished data from the CSSS de Chicoutimi. An ES of at least 0.5 also represents the clinically significant effect needed to justify the pragmatic integration of the intervention into the services already provided. For example, in the case of patient empowerment, the ability to detect an ES of 0.5 translates into the capacity to detect any difference of at least half a point within different domains of the heiQ, since the overall standard deviation is about 1 point . The detection of smaller differences is irrelevant since it is not clinically significant.
Data analysis plan
All statistical analyses will be performed based on an intent-to-treat principle. We will first describe the characteristics of participants in each group, using means and standard deviations (continuous variables) or percentages (categorical variables). The groups will be compared at baseline (T0) using Student’s T test or the Chi-square test. Wherever possible, comparisons will be made between patients who agreed to participate in either phase of the study and those who refused, in order to document biases related to refusals. For outcome indicators at 3 months (self-efficacy, self-management practices, health habits, activation and psychological distress), groups will be compared at T1 using analysis of covariance adjusted for T0 scores. For indicators measured at six months (empowerment, quality of life and use of care), repeated measures analysis of variance will compare change over time in the two groups. In all cases, if the groups initially differ with regard to certain characteristics despite randomization, analysis will be adjusted to take into account the relevant variables. In addition, if a nurse effect is present (non-null intracluster correlation), despite efforts at standardization, multi-level analysis will be conducted to take this into account.
The economic analysis of the intervention will focus on a cost-effectiveness and a cost-benefit analysis. The implemented intervention and the usual care will be compared in each of these analyses.
The cost-effectiveness analysis (CEA) will compare the relative costs invested and effects of implemented intervention and usual care. On the other hand, the cost-benefit analysis will indicate the savings per dollar invested in the implemented intervention in terms of the benefit/cost ratio. Benefits will be assessed by assigning a monetary value to the effects. Because of the ethical and methodological problems in the assignment of a monetary value to quality of life, morbidity and mortality [77, 78], the cost-benefit analysis will be based on a pragmatic approach; it will focus on the savings gained in relation to the reduction of health care use due to the implemented intervention.
Stages of data acquisition and analysis
(1)Cost analysis. Cost analysis will be performed from an organizational perspective; that is, only costs related to the FMG and CSSS will be outlined. The costs of the intervention and of usual care will be identified. For the experimental group, the average nurse’s salary for the time devoted to intervention will be obtained (average cost per patient). The data on services obtained from the CSSS (emergency visits, hospitalizations, etc.) and from the FMG (excluding management of cases already included in the calculation) during the six-month follow-up will also be obtained and their monetary value will be estimated, using the average cost per patient for both groups. (2) Measurement of effectiveness. Data collected as part of the impact evaluation (intermediate outcomes) will be used to document the effects of the intervention on empowerment, quality of life, and use of services (emergency visits, hospitalizations, other CSSS services and FMG services). The effects will be measured in terms of the empowerment gained, quality of life gained, and, the reduction in use of services. (3) Efficiency analysis. Costs and effects will be compared, in order to identify whether the intervention or the usual care leads to better effectiveness at lower cost. Incremental cost-effectiveness ratios (ICER = Δ Effectiveness/Δ Costs) will be calculated. This component will address the following three questions: (1) How much does it cost to improve empowerment? (2) How much does it cost to improve quality of life? (3) How much does it cost to decrease use of CSSS or FMG services? (4) Cost-benefit analysis. A cost-benefit analysis will be performed if a reduction in the use of services is documented (see power calculation in “sample size and statistical power” section presented previously). The cost differential of the cost of services avoided, related to a reduction in the use of services for the number of patients surveyed (Δ benefits), will be compared to the cost differential of the intervention and the usual care (Δ costs) so as to assess whether the new intervention is cost-effective. We will calculate the gain per dollar invested in the intervention by estimating the Δ benefit/Δ cost ratio. If this ratio is greater than 1, a gain will be indicated; otherwise, a break-even point will be estimated by evaluating the minimum cost at which the intervention will become cost-effective. This analysis would also estimate the caseload for obtaining a break-even point and for identifying the profile of patients for whom the intervention was more cost-effective. (5) Sensitivity analysis. Since many uncertainties are generally present in the economic and effectiveness data, it will be important to complete the analysis by: (1) identifying and explaining the sources of uncertainty; and (2) performing sensitivity analyses by varying the value of specific parameters related to the sources of uncertainty, in order to assess the robustness of outcomes [77, 79], and if necessary, to determine the break-even point of the intervention.
This study was approved by the Research Ethics Board (REB) of the CSSS de Chicoutimi for both CSSS. Informed consent will be obtained from all participants. The six-month wait for the intervention for the control group has a negligible effect given the chronic nature of the conditions for which the intervention is proposed. Specific consent will be sought from each participating patient for access to their administrative health data and its use by the parties involved in the study. Confidentiality will be respected and data security ensured according to the rules in force within both CSSS and by the Research Ethics Board of the CSSS de Chicoutimi. Any publication resulting from this research will respect patient confidentiality.
The integration of case management by nurses and of self-management support groups into the FMG has the potential to impact patients positively, as outlined in the logic model. The long-term effects described therein cannot be measured due to the program's short timeframe, but may be confidently assumed from the evidence provided in the literature. FMG nurses will be able to continue to offer the intervention in their FMG after the implementation of the intervention. The caseload numbers that provide an optimal cost-benefit and a positive outcome profile for target patients will inform decision-makers and managers on the human and financial resources required to achieve optimal outcomes. In addition, decision-makers, managers and health care professionals will be aware of the factors to consider that favour the implementation of this intervention in FMG and other CSSS of the region and throughout Quebec.
Strengths and limitations
The study design will not allow us to determine the individual effects of each component (case management and self-management group support) of the intervention. It was conceived in order to evaluate the addition of the self-management component for some patients who can benefit from it according to primary care practitioners’ perspective. Independent applications for funding are planned in order to evaluate the effects of each component in a near future. A contamination bias could occur between the case management nurses and the nurses involved with the patient control group. Several precautions will be taken to minimize this bias. First, no nurse will monitor both experimental group and control group. Discussions between nurses caring for the groups will be kept to a minimum with respect to the intervention and the new follow-up methods developed during the first six months of implementation. The implementation analysis will shed a qualitative light on this phenomenon. The presence of eight case management nurses raises the possibility of a “cluster” effect at the analysis level, which will be verified. However, this should be minimal since many precautions will be taken to reduce it: proper training of eight case management nurses to ensure standardization; the important role of the Clinical Project Coordinator in maintaining a comparable level of intervention among the eight nurses; regular discussions (every 1.5 months) among case management nurses. Repeat surveys induce a learning effect; however, the time-lapse between each survey questionnaire will be sufficient for the effect to be minimal. Regarding external validity, the pragmatic nature of the effects evaluation favours generalization. Analysis of the implementation process will identify the factors to be considered and the conditions to be put into place to support implementation of the intervention in other Quebec FMG.
Centre de santé et de services sociaux / Health and social services centre
Disease Burden Morbidity Assessment
Family medicine group
Hospital Anxiety and Depression Scale
Health Education Impact Questionnaire
Intervention fidelity checklist
Individualized service plan
Promoting Action on Research Implementation in Health Services
Program of research on the evolution of a cohort investigating health system effects
This study is made possible by the financial support of the Pfizer-FRSQ-MSSS chronic disease fund. We would like to acknowledge the extended research team members for their contribution in the conception of the study and in reading and commenting the protocol manuscript. We would like to thank the clinical team members of the four participating FMG for their help and advice with our research project. Furthermore we want to thank the partners from the regional health and social services agency, from the participating CSSS and from the community partners. We would like to thank Susie Bernier, from the CIHR Chair on Health Services and Policy Research on Chronic Diseases in Primary Care, and Valentina Baslyk for their work in the translation and editing of the manuscript.
- Commissaire à la santé et au bien-être du Québec: Rapport d’appréciation de la performance du système de santé et de services sociaux 2010: État de situation portant sur les maladies chroniques et la réponse du système de santé et de services sociaux. 2010, Montréal: Gouvernement du QuébecGoogle Scholar
- Organisation mondiale de la santé: Innovative care for chronic conditions - Building blocks for action: Global report. 2002, Genève: World Health OrganizationGoogle Scholar
- Marks R, Allegrante JP, Lorig K: A review and synthesis of research evidence for self-efficacy-enhancing interventions for reducing chronic disability: implications for health education practice (part II). Health Promot Pract. 2005, 6 (2): 148-156. 10.1177/1524839904266792.View ArticlePubMedGoogle Scholar
- Kreindler SA: Lifting the burden of chronic disease: What has worked? What hasn't? What's next?. Healthc Q. 2009, 12 (2): 30-40.View ArticlePubMedGoogle Scholar
- Departement of Health: Supporting People with Long Term Conditions: An NHS and Social Care Model to Support Local Innovation and Integration. 2005, UK.Google Scholar
- Corser W, Dontje K: Self-management perspectives of heavily comorbid primary care adults. Prof Case Manage. 2011, 16 (1): 6-17.View ArticleGoogle Scholar
- MSSS: Stratégie de prévention et de gestion des maladies chroniques et Plan d’action 2008–2013. 2008, Québec: Ministère de la santé et des services sociauxGoogle Scholar
- McMurphy D: Quels sont les caractéristiques et les avantages fondamentaux de services de santé de première ligne de qualité ?. 2009, Ottawa: FCRSSGoogle Scholar
- Singh D: Transforming chronic care: Evidence about improving care for people with long-term conditions. 2005, Birmingham, UK: University of Birmingham & NHS Institue for innovation and improvementGoogle Scholar
- Khan Y, Glazier RH, Moineddin R, Schull MJ: A population-based study of the association between socioeconomic status and emergency department utilization in Ontario, Canada. Acad Emerg Med. 2011, 18 (8): 836-843. 10.1111/j.1553-2712.2011.01127.x.View ArticlePubMedGoogle Scholar
- Disano J, Goulet J, Muhajarine N, Neudorf C, Harvey J: Socio-economic status and rates of hospital admission for chronic disease in urban Canada. Can Nurse. 2010, 106 (1): 24-29.PubMedGoogle Scholar
- Noël PH, Williams JW, Unützer J, Worchel J, Lee S, Cornell J, Katon W, Harpole LH, Hunkeler E: Depression and comorbid illness in elderly primary care patients: Impact on multiple domains of health status and well-being. Ann Fam Med. 2004, 2 (6): 555-562. 10.1370/afm.143.View ArticlePubMedPubMed CentralGoogle Scholar
- Gott M, Barnes S, Parker C, Payne S, Seamark D, Gariballa S, Small N: Predictors of the quality of life of older people with heart failure recruited from primary care. Age Ageing. 2006, 35 (2): 172-177. 10.1093/ageing/afj040.View ArticlePubMedGoogle Scholar
- Walker AE: Multiple chronic diseases and quality of life: patterns emerging from a large national sample, Australia. Chronic Illn. 2007, 3 (3): 202-218. 10.1177/1742395307081504.View ArticlePubMedGoogle Scholar
- Lemstra M, Mackenbach J, Neudorf C, Nannapaneni U: High health care utilization and costs associated with lower socio-economic status: results from a linked dataset. Can J Public Health. 2009, 100 (3): 180-183.PubMedGoogle Scholar
- Bieler G, Paroz S, Faouzi M, Trueb L, Vaucher P, Althaus F, Daeppen J-B, Bodenmann P: Social and medical vulnerability factors of emergency department frequent users in a universal health insurance system. Acad Emerg Med. 2012, 19 (1): 63-68. 10.1111/j.1553-2712.2011.01246.x.View ArticlePubMedGoogle Scholar
- Althaus F, Paroz S, Hugli O, Ghali WA, Daeppen JB, Peytremann-Bridevaux I, Bodenmann P: Effectiveness of interventions targeting frequent users of emergency departments: a systematic review. Ann Emerg Med. 2011, 58 (1): 41-52. 10.1016/j.annemergmed.2011.03.007. e42View ArticlePubMedGoogle Scholar
- Beaulieu MD, Denis JL, D’Amour D, Goudreau J, Haggerty J, Hudon E, Jobin G, Lamothe L, Gilbert F, Guay H: L’implantation des Groupes de médecine de famille: Un défi de la réorganisation de la pratique et de la collaboration interprofessionnelle. Étude de cas dans cinq GMF de la première vague au Québec. 2006, Ottawa: FCRSSGoogle Scholar
- Family Medicine Groups. http://www.msss.gouv.qc.ca/en/sujets/organisation/gmf.php.
- du Québec G: Évaluation de l'implantation et des effets des premiers groupes de médecine de famille au Québec. 2009, Québec: Direction des communications du ministère de la Santé et des Services sociaux du QuébecGoogle Scholar
- Lawlor P: Referring patients with complex problems to community matrons. Prim Health Care. 2009, 19 (10): 22-24.View ArticleGoogle Scholar
- Chouinard MC, Fortin M, Tremblay S: Table ronde sur les maladies chroniques au Saguenay Saguenay-Lac-Saint-Jean: Constats et recommandations. 2011, Chicoutimi: Agence de santé et de services sociaux du Saguenay–Lac-Saint-JeanGoogle Scholar
- Department of Health: Caring for people with long term conditions: An education framework for community matrons and case managers. 2006, UK: Department of Health - Long term conditions teamGoogle Scholar
- Bilodeau MN: Programme Arrimage: Rapport d'activités. 2011, Chicoutimi: CSSS de ChicoutimiGoogle Scholar
- Schraeder C, Fraser CW, Clark I, Long B, Shelton P, Waldschmidt V, Kucera CL, Lanker WK: Evaluation of a primary care nurse case management intervention for chronically ill community dwelling older people. J Nurs Healthc Chronic Illn. 2008, 17 (11C): 407-417.Google Scholar
- Sutherland D, Hayter M: Structured review: Evaluating the effectiveness of nurse case managers in improving health outcomes in three major chronic diseases. J Clin Nurs. 2009, 18 (21): 2978-2992. 10.1111/j.1365-2702.2009.02900.x.View ArticlePubMedGoogle Scholar
- Kendall S, Wilson P, Procter S, Brooks F, Bunn F, Gage H, McNeilly E: The nursing contribution to chronic disease management: A whole systems approach. 2010, Hatfield, UK: Queen's Printer and Controller of HMSOGoogle Scholar
- Santé Canada: Fond pour l’adaptation des soins de santé primaires: Résumé des initiatives. 2007, Ottawa: Gouvernement du CanadaGoogle Scholar
- Zwar N, Harris M, Griffiths R, Roland M, Dennis S, Powell Davies G: A systematic review of chronic disease management. 2006, Canberra: Australian Primary Health Care Research InstituteGoogle Scholar
- Chodosh J, Morton SC, Mojica W, Maglione M, Suttorp MJ, Hilton L, Rhodes S, Shekelle P: Improving patient care. Meta-analysis: Chronic disease self-management programs for older adults. Ann Intern Med. 2005, 143 (6): 427-438.View ArticlePubMedGoogle Scholar
- Commissaire à la santé et au bien-être du Québec: Rapport d’appréciation de la performance du système de santé et de services sociaux 2009. Construire sur les bases d’une première ligne de soins renouvelée: Recommandations, enjeux et implications. 2009, Montréal: Gouvernement du QuébecGoogle Scholar
- Richardson J, Letts L, Chan D, Stratford P, Hand C, Price D, Hilts L, Coman L, Edwards M, Baptiste S: Rehabilitation in primary care setting for persons with chronic illness: A randomized controlled trial. Prim Health Care Res Dev. 2010, 11: 382-395. 10.1017/S1463423610000113.View ArticleGoogle Scholar
- Harvey PW, Petkov JN, Misan G, Fuller J, Battersby MW, Cayetano TN, Warren K, Holmes P: Self-management support and training for patients with chronic and complex conditions improves health-related behaviour and health outcomes. Aust Health Rev. 2008, 32 (2): 330-338. 10.1071/AH080330.View ArticlePubMedGoogle Scholar
- Williams AM, Dennis S, Harris MF: How effective are the linkages between self-management programmes and primary care providers, especially for disadvantaged patients?. Chronic Illn. 2010, 7 (1): 20-30.View ArticlePubMedGoogle Scholar
- Bodenheimer T, Lorig K, Holman H, Grumbach K: Innovations in primary care. Patient self-management of chronic disease in primary care. JAMA. 2002, 288 (19): 2469-2475. 10.1001/jama.288.19.2469.View ArticlePubMedGoogle Scholar
- Slaughter J: Kaiser Permanente: Integrating around a care delivery model. J Ambulatory Care Manage. 2000, 23 (3): 39-47.View ArticlePubMedGoogle Scholar
- National Health Service: Improving the care for people with long-term conditions. 2006, Birmingham, UK: University of BirminghamGoogle Scholar
- Kitson A, Harvey G, McCormack B: Enabling the implementation of evidence based practice: A conceptual framework. Qual Health Care. 1998, 7 (3): 149-158. 10.1136/qshc.7.3.149.View ArticlePubMedPubMed CentralGoogle Scholar
- Kitson AL, Rycroft-Malone J, Harvey G, McCormack B, Seers K, Titchen A: Evaluating the successful implementation of evidence into practice using the PARiHS framework: theoretical and practical challenges. Implementation Sci: IS. 2008, 3: 1-1. 10.1186/1748-5908-3-1.View ArticlePubMedPubMed CentralGoogle Scholar
- American Nurses Association: Nursing's social policy statement. 2010, Silver Spring: MD: ANAGoogle Scholar
- Freund T, Peters-Klimm F, Rochon J, Mahler C, Gensichen J, Erler A, Beyer M, Baldauf A, Gerlach FM, Szecsenyi J: Primary care practice-based care management for chronically ill patients (PraCMan): study protocol for a cluster randomized controlled trial [ISRCTN56104508]. Trials. 2011, 12: 163-163. 10.1186/1745-6215-12-163.View ArticlePubMedPubMed CentralGoogle Scholar
- Brownson CA, Miller D, Crespo R, Neuner S, Thompson J, Wall JC, Emont S, Fazzone P, Fisher EB, Glasgow RE: A quality improvement tool to assess self-management support in primary care. Joint Comm J Qual Patient Saf. 2007, 33 (7): 408-416.Google Scholar
- Tait AR: Clinical governance in primary care: A literature review. J Clin Nurs. 2004, 13 (6): 723-730. 10.1111/j.1365-2702.2004.00949.x.View ArticlePubMedGoogle Scholar
- Johnston S, Irving H, Mill K, Rowan MS, Liddy C: The patient's voice: an exploratory study of the impact of a group self-management support program. BMC Fam Pract. 2012, 13 (1): 65-72. 10.1186/1471-2296-13-65.View ArticlePubMedPubMed CentralGoogle Scholar
- Champagne F, Contandriopoulos A, Brousselle A, Hartz Z, Denis J: L’évaluation dans le domaine de la santé: Concepts et méthodes. L’évaluation: Concepts et méthodes. Edited by: Brousselle A, Champagne F, Contandriopoulos A, Hartz Z. 2009, Montréal: PUM, 35-56.Google Scholar
- Pawson R, Tilley N: Realistic evaluation. 1997, London: SageGoogle Scholar
- Walsh K, Duke J, Foureur M, Macdonald L: Designing an effective evaluation plan: a tool for understanding and planning evaluations for complex nursing contexts. Contemp Nurse. 2007, 25 (1–2): 136-145.View ArticlePubMedGoogle Scholar
- Ridde V, Dagenais C: Introduction générale à l’évaluation de programme. Approches et pratiques en évaluation de programme. Edited by: Ridde V, Dagenais C. 2009, Montréal: PUM, 11-29.Google Scholar
- Yin RK: Case study research: Design and methods. 2008, Thousand Oaks, CA: SageGoogle Scholar
- Champagne F, Brousselle A, Hartz Z, Contandriopoulos A, Denis J: L’analyse de l’implantation. L'évaluation: Concepts et méthodes. Edited by: Brousselle A, Champagne F, Contandriopoulos A, Hartz Z. 2009, Montréal: PUM, 225-250.Google Scholar
- Koenig G: Realistic evaluation and case studies: Stretching the potential. Evaluation. 2009, 15 (1): 9-30. 10.1177/1356389008097869.View ArticleGoogle Scholar
- Sidani S, Braden SJ: Intervention fidelity. Design, evaluation, and translation of nursing interventions. Edited by: Sidani S, Braden SJ. 2011, San Francisco, CA: John Wiley & Sons, 125-145.View ArticleGoogle Scholar
- Gagnon YC: L'étude de cas comme méthode de recherche. 2011, Québec: PUQ, 2Google Scholar
- Lincoln YS, Guba E: Naturalistic enquiry. 1985, Beverley Hills, CA: SageGoogle Scholar
- MacPherson H: Pragmatic clinical trials. Complement Ther Med. 2004, 12 (2–3): 136-140.View ArticlePubMedGoogle Scholar
- Fortin MF: Fondements et étapes du processus de recherche. 2010, Montréal: ChenelièreGoogle Scholar
- Freund T, Mahler C, Erler A, Gensichen J, Ose D, Szecsenyi J, Peters-Klimm F: Identification of patients likely to benefit from care management programs. Am J Manag Care. 2011, 17 (5): 345-352.PubMedGoogle Scholar
- Médiamed Technologies: Mesure de la performance: Logiciel MAGIC Chronique. Mont-Saint-Hilaire, Québec: Médiamed Technologies
- Moher D, Hopewell S, Schulz KF, Montori V, Gøtzsche PC, Devereaux PJ, Elbourne D, Egger M, Altman DG: CONSORT 2010 Explanation and Elaboration: Updated guidelines for reporting parallel group randomised trials. J Clin Epidemiol. 2010, 63 (8): e1-e37. 10.1016/j.jclinepi.2010.03.004.View ArticlePubMedGoogle Scholar
- Steed L, Mulligan K, Newman S: Development and evaluation of self-management interventions. Chronic physical illness: Self-management and behavioral interventions. Edited by: Newman S, Steed L, Mulligan K. 2009, Beckshire, UK: Open University Press, 135-147.Google Scholar
- Lorig KR, Sobel DS, Ritter PL, Laurent D, Hobbs M: Effect of a self-management program on patients with chronic disease. Eff Clin Pract. 2001, 4 (6): 256-262.PubMedGoogle Scholar
- Osborne RH, Elsworth GR, Whitfield K: The Health Education Impact Questionnaire (heiQ): An outcomes and evaluation measure for patient education and self-management interventions for people with chronic conditions. Patient Educ Couns. 2007, 66 (2): 192-201. 10.1016/j.pec.2006.12.002.View ArticlePubMedGoogle Scholar
- Osborne RH, Batterham R, Livingston J: The evaluation of chronic disease self-management support across settings: the international experience of the health education impact questionnaire quality monitoring system. Nurs Clin North Am. 2011, 46 (3): 255-10.1016/j.cnur.2011.05.010.View ArticlePubMedGoogle Scholar
- Haggerty J, Fortin M, Beaulieu M-D, Hudon C, Loignon C, Préville M, Roberge D: At the interface of community and healthcare systems: a longitudinal cohort study on evolving health and the impact of primary healthcare from the patient's perspective. BMC Health Serv Res. 2010, 10: 258-258. 10.1186/1472-6963-10-258.View ArticlePubMedPubMed CentralGoogle Scholar
- Skolasky RL, Green AF, Scharfstein D, Boult C, Reider L, Wegener ST: Psychometric properties of the patient activation measure among multimorbid older adults. Health Serv Res. 2011, 46 (2): 457-478. 10.1111/j.1475-6773.2010.01210.x.View ArticlePubMedPubMed CentralGoogle Scholar
- Kessler RC, Andrews G, Colpe LJ, Hiripi E, Mroczek DK, Normand SLT, Walters EE, Zaslavsky AM: Short screening scales to monitor population prevalences and trends in non-specific psychological distress. Psychol Med. 2002, 32 (6): 959-976. 10.1017/S0033291702006074.View ArticlePubMedGoogle Scholar
- Ware J, Kosinski M, Keller SD: A 12-Item Short-Form Health Survey: Construction of scales and preliminary tests of reliability and validity. Med Care. 1996, 34 (3): 220-233. 10.1097/00005650-199603000-00003.View ArticlePubMedGoogle Scholar
- Essink-Bot ML, Krabbe PF, Bonsel GJ, Aaronson NK: An empirical comparison of four generic health status measures. Med Care. 1997, 35 (5): 522-537. 10.1097/00005650-199705000-00008.View ArticlePubMedGoogle Scholar
- Weiss BD, Mays MZ, Martz W, Castro KM, DeWalt DA, Pignone MP, Mockbee J, Hale FA: Quick assessment of literacy in primary care: The newest vital sign. Ann Fam Med. 2005, 3 (6): 514-522. 10.1370/afm.405.View ArticlePubMedPubMed CentralGoogle Scholar
- Zigmond AS, Snaith RP: The Hospital Anxiety and Depression Scale. Acta Psychiatr Scand. 1983, 67 (6): 361-370. 10.1111/j.1600-0447.1983.tb09716.x.View ArticlePubMedGoogle Scholar
- Bjelland I, Dahl AA, Haug TT, Neckelmann D: The validity of the Hospital Anxiety and Depression Scale. An updated literature review. J Psychosom Res. 2002, 52 (2): 69-77. 10.1016/S0022-3999(01)00296-3.View ArticlePubMedGoogle Scholar
- Bayliss EA, Ellis JL, Steiner JF: Subjective assessments of comorbidity correlate with quality of life health outcomes: initial validation of a comorbidity assessment instrument. Health Qual Life Outcomes. 2005, 3: 51-51. 10.1186/1477-7525-3-51.View ArticlePubMedPubMed CentralGoogle Scholar
- Cohen J: Statistical power analysis for the behavioural sciences. 1988, Illsdale, NJ: Academic Press, 2Google Scholar
- Cochran WG: Sampling techniques. 1977, New York: John Wiley & Sons, 3Google Scholar
- Shumway M, Boccellari A, O'Brien K, Okin RL: Cost-effectiveness of clinical case management for ED frequent users: Results of a randomized trial. Am J Emerg Med. 2008, 26 (2): 155-164. 10.1016/j.ajem.2007.04.021.View ArticlePubMedGoogle Scholar
- Stone GR, Packer TL: Evaluation of a rural chronic disease self-management program. Rural Remote Health. 2010, 10 (1): 10p.Google Scholar
- Drummond MF, Sculpher MJ, Torrance MJ, O’Brien BJ, Stoddart GL: Methods for the economic evaluation of health care programmes 3rd edn. 2005, Toronto: Oxford University PressGoogle Scholar
- Kelman S: Cost-benefit analysis: an ethical critique. Regulation. 1981, 5 (1): 33-40.PubMedGoogle Scholar
- Hansjürgens B: Economic valuation through cost-benefit analysis–possibilities and limitations. Toxicology. 2004, 205 (3): 241-252. 10.1016/j.tox.2004.06.054.View ArticlePubMedGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6963/13/49/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.