Skip to main content
  • Research article
  • Open access
  • Published:

Empirical analysis shows reduced cost data collection may be an efficient method in economic clinical trials

Abstract

Background

Data collection for economic evaluation alongside clinical trials is burdensome and cost-intensive. Limiting both the frequency of data collection and recall periods can solve the problem. As a consequence, gaps in survey periods arise and must be filled appropriately. The aims of our study are to assess the validity of incomplete cost data collection and define suitable resource categories.

Methods

In the randomised KORINNA study, cost data from 234 elderly patients were collected quarterly over a 1-year period. Different strategies for incomplete data collection were compared with complete data collection. The sample size calculation was modified in response to elasticity of variance.

Results

Resource categories suitable for incomplete data collection were physiotherapy, ambulatory clinic in hospital, medication, consultations, outpatient nursing service and paid household help.

Cost estimation from complete and incomplete data collection showed no difference when omitting information from one quarter. When omitting information from two quarters, costs were underestimated by 3.9% to 4.6%.

With respect to the observed increased standard deviation, a larger sample size would be required, increased by 3%. Nevertheless, more time was saved than extra time would be required for additional patients.

Conclusion

Cost data can be collected efficiently by reducing the frequency of data collection. This can be achieved by incomplete data collection for shortened periods or complete data collection by extending recall windows. In our analysis, cost estimates per year for ambulatory healthcare and non-healthcare services in terms of three data collections was as valid and accurate as a four complete data collections. In contrast, data on hospitalisation, rehabilitation stays and care insurance benefits should be collected for the entire target period, using extended recall windows. When applying the method of incomplete data collection, sample size calculation has to be modified because of the increased standard deviation. This approach is suitable to enable economic evaluation with lower costs to both study participants and investigators.

Trial registration

The trial registration number is ISRCTN02893746

Peer Review reports

Background

Economic evaluation alongside clinical trials is gaining importance because of demographic trends towards an ageing population, which is a relevant driver for resource use in healthcare [1–3]. There are different types of data that can be used to collect information on healthcare costs: reviewing medical charts or administrative data sets; drawing on experts’ experiences of healthcare utilisation; and gathering patients’ self-reported resource use [4]. Patients’ self-reported resource use can be recorded prospectively with cost diaries or retrospectively by means of questionnaires. However, cost data collection in clinical trials can be burdensome and can increase both the costs of clinical trials [5] and the burden on study participants [6] and clinical investigators [7].

For economic evaluation from a societal perspective, only patients’ self-reported resource use offers an account of out-of-pocket payments (e.g. paid household help or over-the-counter medications). Cost data collection from health insurance companies is complex because patients are enrolled in different insurance funds, which is true in Germany and also in the USA. Hence, prospective or retrospective methods based on patient recall are widely used [8–10].

Retrospective assessment of costs via questionnaires raises the question of how often participants should be contacted and what is the appropriate recall time frame [11, 12]. Recall ability declines with age, the frequency of resource use and the length of the recall time frame [9]. If patients are asked more frequently, recall bias is minimised, but the cost of data collection and the burden on study participants are increased [13]. For the prospective assessment of costs via cost diaries, the period covered by each diary and the frequency with which the diaries are sent back needs to be defined. For long-term clinical trials, e.g. 12 months or longer, the burden on study participants of keeping the diary is substantial and it is unlikely that participants will complete all the diaries [14].

With regard to the above-mentioned problems, an alternative approach to minimise recall bias and the frequency of questioning is to limit the time period covered by each diary or to limit the recall period [15]. As a consequence, gaps in survey periods arise and must be filled appropriately by the collected data. This approach has already been used to collect cost data in both cross-sectional studies and randomised controlled trials. In a cross-sectional study on healthcare costs in the elderly, Heinrich et al. assessed resource use over various time spans and extrapolated the data to annual costs by assuming that the documented resource use over abbreviated periods would also be found over a 12-month period [16]. In the randomised controlled trial on the cost-utility of psychological treatment for depression and anxiety, Hakkaart-van Roijen et al. [17] interviewed patients every 3 months over a period of 1.5 years asking for medical resource use over the past 4 weeks. They extrapolated the documented resource use by assuming that 4 weeks was representative of the total period of 3 months. Kimman et al. [18] conducted an economic evaluation alongside the randomised controlled trial on treatment for breast cancer, collecting cost data after 3, 6 and 12 months for a period of 4 weeks to interpolate them by assuming that 4 weeks was representative of the in-between periods.

Goossens et al. [14] investigated whether there is a difference in costs between data from limited time periods extrapolated to 1 year and cost diary data from the entire year. They could not find any significant difference between the cost data in the alternative periods. Clarke et al. [13] published a statistical framework specifying cases for which the recall period can be limited by trading off recall bias against information loss.

Considering the importance of limiting the recall period or the period covered by each diary in favour of reduced time and effort for both participants and investigators, there is still little empirical evidence regarding the validity of cost collection with gaps in survey periods in long-term clinical trials. For this purpose, we compared the impact of different gaps in survey periods on the validity of cost measurement in the relevant healthcare service categories using data from a randomised controlled trial.

The main objective of our paper is to assess the validity of incomplete cost data collection concerning the precision and accuracy of cost estimates. Two further objectives are to identify the healthcare service categories suitable or not suitable for inter-/extrapolating incompletely collected cost data and to demonstrate the consequences of collecting data for an abbreviated time period on sample size calculation with respect to changing the variance of the estimate.

Methods

Study design and study population

The data used were obtained from the randomised controlled KORINNA trial (Koronarinfarkt-Nachbehandlung im Alter), which is an ongoing monocentre study at the Central Hospital of Augsburg, Germany, to evaluate the cost-effectiveness of a case management intervention by trained nurses in elderly patients (≥ 65 years) with acute myocardial infarction. The control group received usual care. The primary goal of the KORINNA study is to assess whether case management can reduce readmission or out-of-hospital death. As a secondary objective, the incremental cost-utility ratio of this case management intervention will be estimated. The intervention and observation period covers 1 year. The study protocol, which was approved by the Ethics Committee of the Bavarian Chamber of Physicians, describes the intervention in more detail and is published elsewhere [19]. The trial registration number is ISRCTN02893746. Between September 2008 and May 2010, 340 patients from the Augsburg region were enrolled. For this analysis, cost data for 234 patients with complete 1 year follow-up (quarters one, two, three and four) were available. The mean age of participants was 75 years, and nearly 38% (n=88) were women.

Cost measurement

Cost measurement by bottom-up gross costing [20] is a two-step process. First, duration and/or frequency of resources used by patients were collected via a questionnaire-based interview. Second, unit prices are determined and multiplied by duration and/or frequency of resource use.

Cost measurement was performed from the societal perspective, which includes both direct healthcare costs and direct non-healthcare costs. Indirect costs are not considered in this analysis, as all participants were already retired and did not incur production loss due to illness-related absence from work. Data collection was not limited to disease-related services and protocol-driven costs were factored out [19, 21].

Collection of resource use

Over a 1-year period, participants were interviewed quarterly regarding the previous time period of 3 months, either by computer-assisted telephone interview (CATI) or in a face-to-face interview. In the case of CATI, plausibility checks were included and, in the case of face-to-face interviews, double data entry was applied. Data evaluation and necessary adjustments were performed as described below.

Direct healthcare use

For costs incurred for outpatient and inpatient care, the number of visits to a general practitioner, specialist care, ambulatory clinics in hospital and physiotherapy, days spent in hospitals, intensive care units and rehabilitation were documented. Medication administered at the time of the survey was recorded by IDOM software, a database-supported identification system that logs the name, units, pharmaceutical identification number, time period, package size and price per package [22]. In certain cases, medication data had to be adjusted to calculate the cost of medication properly and this is described as follows. For some medications – such as insulin – package size was converted from millilitres or prefilled pens/syringes to international units. Similar problems arose in the case of drops, inhalants and aerosols. In these cases, the numbers of drops or puffs per package had to be calculated based on information from the pharmaceutical company because the units per package disagree with the units of medication administration. If patients ingested antibacterials or subcutaneously administered anticoagulants, it was assumed that one package of the medication was bought. Medications not taken regularly were not included in the analysis.

Direct non-healthcare use

Costs incurred for formal care and home help were documented as days per week and hours per day of outpatient nursing service and paid household help. The level of care needs, as assessed by the medical services of the long-term care insurance fund, served as a proxy for the extent of informal care [23].

Unit prices for resource use

Unit price calculation for resource use was primarily based on the methods published by Krauth et al. [24]. All unit prices were reported for the year 2008. Table 1 gives an overview of prices assigned to the resource quantities.

Table 1 Unit prices (all unit prices are expressed in euros at 2008 values)

The costs of physicians, published as weighted average values of patient-physician contact for privately and statutorily insured patients, [24] were updated concerning the reimbursement from the Statutory Health Insurance (SHI) per case, drawing on data from the National Association of Statutory Health Insurance Physicians [25]. The weighted costs per contact with physiotherapists (€13.75) were calculated from data from the SHI contract for the supply of remedies [26] and the stipulation of allowance for civil servants [27]. Costs of ambulatory clinics in hospital were calculated by the average reimbursement from the Augsburg hospital (€40.31). The price per hospital day (€511.07) was based on Federal Statistical Office [28] and National Health Report data [29], using Krauth’s [24] calculation method. The latest published price of intensive care units from 2003 [30] was extrapolated to 2008 using the yearly inflation rate of hospital costs [29] (€1,199.14). Inpatient rehabilitation (€100 per day) and outpatient rehabilitation (€1535 per case) were based on data from the pension insurance companies [31]. The prices of medications were based on the medication database of the scientific institute of the statutory sickness funds in Germany (WIDO) [32]. The total drug costs were computed by calculating cost per dose and multiplying by the frequency and duration of administration. Costs per hour of outpatient nursing service (€28.30) and paid household help (€15.70) were valued based on the contract between the associations of nursing caregivers and the Local Health Care Fund [33]. Informal care was valued according to the benefits received per nursing care level, which is granted by the German Long Term Care Insurance after assessing the required level of nursing care [23]. The benefit is a transfer payment to patients, who do not engage outpatient nursing services but also need help in activities of daily living (ADL), e.g. eating, bathing, dressing, and/or in instrumental activities of daily living (IADL), e.g. housework, cooking, shopping. The benefit level is determined by nursing care level and, although it is a transfer payment, it is used here as a proxy value for informal care.

Dealing with missing data

As only data from complete follow-up (quarters one, two, three and four) were used, there were only a few itemwise missing values (96.2% of the patients had complete cost data), except for medication. Hence, for missing values mean imputation was employed by generating means of the observed data for the corresponding variables. Some 16% of medications did not have a valid pharmaceutical identification number. As unit costs are only available for medication with a pharmaceutical identification number, missing data were substituted based on the documented medication name. If no dosage was specified, it was replaced by the patient’s medication data from other quarters.

Impact of different time periods on measurement of resource use and cost

The objective of our analysis was to investigate the validity of incomplete cost data collection in long-term clinical trials, using data from the KORINNA study. We did not consider excluding the first and the last quarter of data collection. The first quarter is important because response rates commonly dwindle after the initial assessment as a result of withdrawal of informed consent or death of study participants and, in the last period, cost data can be collected efficiently together with the final assessments. Accordingly, either the second and/or third quarter were excluded for our study.

For each resource category, mean costs per quarter and per capita were calculated based on quarterly data collected from 1 year. The difference between quarterly costs over the course of time was compared by repeated measures analysis of variance and Tukey’s multiple comparison test. To determine the resource category suitable for incomplete data collection, mean differences between quarters were assessed, and assumed inherent recall bias was considered. In the case of minor recall bias, complete data collection by extending recall windows was preferred to incomplete data collection.

For resource categories suitable for incomplete data collection, different kinds of cost calculations were employed. For extrapolation, the complete values from the following quarter replaced those from the missing quarter. For interpolation, the means of the individual values from the previous quarter and the following quarter were used to replace the values from the missing quarter.

For each method (extrapolation and interpolation), three alternative periods of data collection were considered (Figure 1):

  • Alternative 1 (Alt 1): Data collection in the first, third and fourth quarters and replacing the data from the second quarter by extrapolating the data from the third quarter

  • Alternative 2 (Alt 2): Data collection in the first, second and fourth quarters and replacing the data from the third quarter by extrapolating the data from the fourth quarter

  • Alternative 3 (Alt 3): Data collection in the first and fourth quarters and replacing the data from the second and third quarters by extrapolating the data from the fourth quarter

  • Alternative 4 (Alt 4): Data collection in the first, third and fourth quarters and replacing the data from the second quarter by interpolating the data from the first and third quarters

  • Alternative 5 (Alt 5): Data collection in the first, second and fourth quarters and replacing the data from the third quarter by interpolating the data from the second and fourth quarters

  • Alternative 6 (Alt 6): Data collection in the first and fourth quarters and replacing the data from the second and third quarters by interpolating the data from the first and fourth quarters

Figure 1
figure 1

Overview of incomplete cost collection: methods and omitted quarters. Alternative 0: complete cost collection. Alternative 1: extrapolation, omitted quarter 2, replaced by quarter 3. Alternative 2: extrapolation, omitted quarter 3, replaced by quarter 4. Alternative 3: extrapolation, omitted quarters 2 and 3, replaced by quarter 4. Alternative 4: interpolation, omitted quarter 2, replaced by quarters 1 and 3. Alternative 5: interpolation, omitted quarter 3, replaced by quarters 2 and 4. Alternative 6: interpolation, omitted quarters 2 and 3, replaced by quarter 1 and 4.

We analysed the influence of the chosen methods on the accuracy and precision of the total cost estimate in the following way.

Cost data were calculated as mean costs, and the differences between complete and incomplete cost data collection were tested using standard errors and p-values from paired t-tests. Alternatively, bootstrap resampling was applied to compute p-values, as there was the possibility that the differences in costs were skewed. We drew 1000 re-samples of the data set with replacement.

The effects of alternative intensity of data collection on precision were studied by comparing the standard deviation of the respective estimates.

Larger sample sizes are required because of an increase in the variance of the estimate arising from a shorter period of data collection to keep the power. This effort must be weighed against the advantages of reducing the data collection costs and the burden on study participants.

Based on the equations for sample size calculation for a single outcome and for cost-effectiveness, [34] the impact of a 1% increase in standard deviation on percentage change in sample size, called elasticity (ε), was calculated assuming ceteris paribus, where α, β and the expected mean difference in costs are not changed The details of the mathematical calculations are shown in Appendix 1. The amount of time that would be saved by collecting data only for a limited period is compared with the consumed time for additional patients as a result of the requirements of a larger sample size.

All analyses were performed with SAS (Version 9.2, SAS-Institute Inc., Cary, NC, USA).

Results

Resource use

Table 2 provides an overview of mean resource use per patient over 12 months.

Table 2 Mean resource use (in number of contacts unless stated otherwise) per patient over 12 months, n=234

Within 1 year, almost all subjects consulted a general practitioner and ca. 68% consulted a specialist in internal medicine. On average, patients consulted a physician 21 times a year. All patients were on medication and ingested on average seven different types of medication at the same time. Approximately one third of subjects received physiotherapy about 16 times over the year and attended the ambulatory clinic at hospital about three times a year. Forty-five per cent of patients were hospitalised and admitted for an average of 19 days; about one third of them had a stay in the intensive care unit of 3.6 days on average. The majority (59%) of patients received rehabilitation. About 8% used the outpatient nursing service for 1.3 h per week, and ca. 11% employed paid household help for 4.8 h per week. Approximately 93% of the patients were without a care level; about 6% were classed as care level one, 1% as care level two and none as care level three.

Resource categories suitable for incomplete data collection

As presented in Table 3, the Tukey test showed that the costs of inpatient care, rehabilitation and care insurance benefit levels as a proxy for informal care were significantly different between quarters. Usage of physiotherapy, ambulatory clinic in hospital, outpatient nursing service and paid household help did not differ significantly between quarters (Table 3). Costs of physicians were significantly lower in the first quarter than in subsequent quarters. Costs of medication were significantly lower in the fourth quarter than in the second and third quarters.

Table 3 Mean costs per patient over different quarters, n=234

Validity of incomplete cost collection

For the cost categories physiotherapy, ambulatory clinic in hospital, outpatient nursing service, paid household help, visits to physicians and medications, complete cost collection was compared with incomplete cost collection to ascertain which period of data collection could be omitted.

Comparisons of complete cost collection per year and per capita vs. three alternative periods of incomplete data collection using extrapolation are illustrated in Table 4.

Table 4 Complete cost collection vs. incomplete cost collection and extrapolation (by the respective quarter), n=234

All comparisons between complete and incomplete data collection were not significantly different regarding the p-values of paired t-tests or the p-values of bootstrapping the differences in costs. Only drug costs, which averaged €1315.52 in complete data collection, differed significantly from incomplete data collection when applying Alt 3 (€1264.26) and Alt 2 (€1291.31). That was to be expected as the mean costs of the second and third quarters differed significantly from the mean costs of the fourth quarter (Table 3). By extrapolating the costs of the third quarter to the second quarter (Alt 1), cost estimation per year showed no significant differences in means.

Comparisons of complete and incomplete data collection using interpolation are shown in Table 5.

Table 5 Complete cost collection vs. incomplete cost collection and interpolation (by the respective quarters), n=234

In the case of incomplete data collection and interpolation, there were three significant differences in comparison with complete data collection. Drug costs differed when applying Alt 5 and Alt 6; costs of ambulatory clinics in hospital differed when applying Alt 6; the sum of direct healthcare costs differed when applying Alt 4. In addition, most of the differences between complete and incomplete cost collection were larger than the differences from extrapolation.

Total cost estimation from incomplete data collection in the case of two data collections was significantly lower independent of whether extrapolation (3.9%) or interpolation (4.6%) was used. Interpolation and extrapolation in the case of omitting information from the second or third quarter showed no difference.

Table 6 illustrates the precision regarding change in standard deviation due to incomplete data collection. In some cases, we found smaller and, in some cases, larger standard deviations than in complete data collection. For the sum of direct healthcare costs, the largest increase was 6.45%, for the sum of direct non-healthcare costs 1.65%, and for the sum altogether 1.65%.

Table 6 Percentage change in variance of the estimate: complete cost collection vs. incomplete cost collection

The consequence on sample size calculation

Incomplete data collection will universally lead to increasing variance of the estimate [13, 14]. As the variance and standard deviation influence sample size calculation, we modelled the impact of a 1% increase in standard deviation on sample size, called elasticity (ε). Mathematical calculations are shown in detail in Appendix 1. In the case of a single outcome (mean difference in costs), the sample size has to increase by 2% if the standard deviation increases by 1%. In the case of cost-effectiveness research, the increase in sample size is smaller. If the standard deviation increases by 1%, the sample size has to increase by 2 b d + b % . As b >0 and d >0, the elasticity for cost-effectiveness is lower than for a single outcome.

For the sum of costs, the largest increase was 1.65% in the case of Alt 5. This would lead to an increase in sample size of 3.3%, i.e. about 12 patients (from planned 340 to 352).

The necessary time to conduct the quarterly interview including set-up time was estimated at 0.5 h. The time for initial and final assessment was estimated at 3 h per assessment. Incomplete data collection would save between 170 h (one omitted quarter) and 340 h (two omitted quarters) compared with 84 h, which are still required for two quarterly interviews and initial and final assessment for additional patients.

Discussion

The aims of this paper were to define suitable resource categories for incomplete cost data collection, to assess the validity of alternative strategies, and to point out the consequences with respect to efficiency.

To determine the resource category suitable for incomplete data collection, inherent recall bias and mean differences between quarters should be assessed. In the case of minor recall bias, Clarke et al. recommend only the collection of complete data by extending recall windows to ensure that no information is lost and data are still collected less frequently [13]. Applied to our study, this means that data collection in the first quarter for the previous time period of 3 months and in the fourth quarter for the previous 9 months would be preferable to incomplete data collection. Recall may be influenced by the frequency and severity of events, [9, 11] so that less frequent visits and more serious events improve the memory of resource use.

It became apparent that it is not appropriate to collect data on hospitalisation, rehabilitation and care insurance benefits using an incomplete algorithm, but to account for the entire target period by using longer recall periods. These cost categories were both relevantly different between quarters and the associated events were less frequent or more serious. Inpatient care indicates a serious event, and patients who had at least one admission to hospital had on average 1.9 stays per year (Table 2). As rehabilitation only occurs once a year, it is a rare event; only five patients reported two stays per year. Care insurance benefits are granted by care insurance funds, following a lengthy needs assessment process. For these cases, recall bias is hardly likely. Bhandari et al. also stated in their review that hospitalisation is a salient event that may be gathered accurately by applying longer recall periods [9] without an appreciable increase in recall bias. In addition, patient reports on hospital stays could be cross-validated against electronic hospital records where these exist. Clarke et al. recommended trading off recall bias against information loss, which would be caused by incomplete data collection. Only if the degree of variation introduced by incomplete data collection is smaller than the bias introduced by recall error should a short time period for cost assessment be preferred [13].

Attention should be paid to the fact that care insurance benefit is a proxy for informal care. If costs incurred for informal care are determined by the hours of care provided by relatives, neighbours or friends, incomplete data collection can be used in a similar way to the collection of outpatient nursing services or paid household help.

The resource categories physiotherapy, ambulatory clinic in hospital, medication, consultations (physicians), outpatient nursing service and paid household help are deemed to be appropriate for incomplete data collection for several reasons. First, there is no evidence for differences in mean costs between the middle quarters (quarters two and three) (Table 3). Second, the portion of these costs is about 30% of total costs, so that an inaccuracy of 5% causes a difference of only 1.5%. The main cost-driving events are hospitalisation and rehabilitation accounting for about 70%. Ridyard et al. [8] advise in their systematic review the balancing of resource use data collection between the main cost driving events, the frequency of data collection and the burden on the researcher. Third, existing differences in mean costs between quarter one and all the other quarters do not become important in the case of extrapolation. Although cost data from quarter one are employed in the case of interpolation, only the sum of direct healthcare costs differed significantly from complete cost collection (Table 5, Alt 4) when data were collected three times.

The lower costs in the resource categories physiotherapy, ambulatory clinic in hospital, medication, consultations (physicians), outpatient nursing service, and paid household help in quarter one (Table 3) can be explained by increased use of rehabilitation and hospital admissions during this time, which supplants ambulatory resource use.

Significant differences in mean costs between quarter four and quarters two or three occur in the case of medication data (Table 3). This reflects the drug regimen guidelines [35, 36] for the acute coronary syndrome, which recommend the use of clopidogrel, an antiplatelet drug, as follow-up treatment for up to 9 months only.

In our analysis, extrapolation turned out to be the better instrument for replacing the omitted periods, as quarter one showed consistently lower costs. Data from quarter one were not used for extrapolation, but for interpolation. Therefore, mean differences between complete and incomplete cost collection from extrapolation were mostly smaller than differences from interpolation (Tables 4 and 5). Furthermore, medication data proved to be only suitable for omitting quarter two and replacing by quarter three, as quarter four is not a representative quarter.

A comparison of our results is restricted by the lack of publications of empirical analyses regarding incomplete data collection. In a study of 174 patients with a stable chronic disease (fibromyalgia and low back pain), Goossens et al. showed no significant differences between multiple time frames of incomplete data collection [14]. Thus, the authors concluded that, for patients with chronic diseases, incomplete cost data collection poses no problem for economic evaluations. They compared the differences in median by Wilcoxon’s signed rank test but not in arithmetic means, and they did not distinguish between different cost categories. As the comparison of means is central to any economic evaluation, non-parametric tests that address differences in the median and analyses of log-transformed costs that address differences in the geometric means are not well suited for this purpose [37]. However, in comparison with our study, the sample size of 174 participants was even smaller, which poses severe limitations with regard to the statistical power of the analysis.

Nevertheless, the authors indicated that, in the case of acute diseases, randomised clinical trials and chronic diseases with seasonal effects, the necessary assumptions of agreement between the different time periods could not be met. Clarke et al. [13] argued that irregular consumption patterns add estimation error. Seasonal effects only become important if the time frame of recruitment is relatively short. If recruitment or the start of the intervention cover a 1-year period, seasonal effects occur for individuals but not for groups. As the comparison of arithmetic means between groups is central to any economic evaluation, group estimates have to be valid, but the results for individuals may differ from each other [15]. Our analysis shows that, in the case of acute myocardial infarction with a 1-year follow-up, several kinds of resource categories are more appropriate for incomplete cost data collection than others. Generalisation of our findings is limited to the elderly population with acute diseases, followed by a chronic course associated with a continuous treatment scheme, as patients with acute myocardial infarction have similar patterns in the long-term course of disease and treatment. Applying incomplete data collection, several points have to be considered when choosing the method (inter- or extrapolation) and omitting quarters. Only those periods can be omitted for which it can be assumed that they are representative of other periods. Equally, only periods for which one may assume that they represented omitted periods can be used to replace the omitted periods. For other studies, the choice of omitted periods may depend on the disease and the intervention so that these assumptions must be tested in pilot studies or based on expert opinions or literature research.

It is important to reduce the burden on study participants, especially in older participants, by decreasing the frequency of data collection. Because this can be achieved by incomplete data collection or extending recall windows, one should carefully consider and differentiate which time frame and method of cost data collection are appropriate for the respective resource category. As an example, Heinrich et al. [16] assessed resource use by employing different time frames for the respective resource category, whereas Hakkaart-van Roijen et al. [17] and Kimman et al. [18] did not distinguish between different resource categories.

A further problem resulting from incomplete cost collection arises from the withdrawal of informed consent or death of the study participants, as it can be assumed that missing data increase because of longer time periods between data collection. For this reason, we recommend not omitting the first quarter.

Although incomplete cost collection will universally lead to increasing variance of the estimate, [13, 14] we only found partially larger standard deviations than in complete data collection. Goossens et al. exclusively found smaller standard deviations, which they attributed to random error, and they recommended including ‘more’ patients [14]. To our knowledge, no calculation concerning increasing standard deviations and sample size has been published so far. When we assume that a suitable resource category will be collected incompletely, our estimate requires a larger sample size of about 3% at most. Nevertheless, more time would be saved as a result of incomplete data collection than extra time required for assessing additional patients. Furthermore, the burden on study participants and clinical investigators can be diminished through the economic data collection effort. When conducting economic analysis alongside clinical trials by means of incomplete data collection, sample size calculation has to be modified.

Conclusions

In economic evaluation, cost data can be collected efficiently by reducing the frequency of data collection. This can be achieved by data collection for shorter periods, by implication incomplete data collection, or extending recall windows so that data are collected completely. To minimise bias by recall error or information loss, one has to consider carefully which resource category is suitable for incomplete data collection. In our analysis, cost estimates per year for ambulatory healthcare and non-healthcare services in terms of three data collections was as valid and accurate as a four complete data collections. Applying incomplete data collection, it should be considered that only periods are suitable to be omitted that have similarity to other periods. The choice of the periods depends on the disease, treatment guidelines and the intervention.

When using the method of incomplete data collection sample, size calculation has to be modified because of increased variation. This approach is suitable to lower the burden and costs for the study participants and investigators in economic evaluation alongside clinical trials. Further empirical analysis regarding the validity of incomplete cost collection must be performed in order to improve the already existing practice of incomplete cost data collection.

Appendix 1

The equation for sample size calculation for a single outcome, thus as costs (or effects) between groups is [33]

n > z α / 2 + z β 2 σ CT 2 + σ CC 2 Δ E ˜ 2

where α type I error, the probability of falsely rejecting the null hypothesis when in fact a true difference exists z α / 2 standardised normal deviate, such for α=0.05, z α / 2 (two-sided)=1.96β type II error, the probability of not rejecting the null hypothesis when it is false z β standardised normal deviate, such for β=0.10, z β (one-sided)=1.28 σ CT 2 variance of the costs in the intervention (trial) group σ CC 2 variance of the costs in the control group Δ Ε ˜ difference in costs between two groupsTo simplify the equation: z α / 2 + z β 2 = a 2 σ CT 2 + σ CC 2 = b 2 Δ Ε ˜ 2 = c 2

n > a 2 b 2 c 2

The equation for ε is

ε = ∂ f ∂ b ⋅ b f
= 2 b a 2 c 2 â‹… b a 2 b 2 c 2

=2If the standard deviation increases by 1 percent, the sample size has to increase by 2 percent.The equation for sample size calculation for cost-effectiveness is [33]

n > z α / 2 + z β R c σ ET 2 + σ CT 2 + σ CC 2 Δ C ˜ − R c Δ E ˜ 2

where σ ET 2 variance of the effects in the intervention (trial) group σ EC 2 variance of the effects in the control group σ CT 2 variance of the costs in the intervention (trial) group σ CC 2 variance of the costs in the control group Δ C ˜ difference in costs between two groups Δ E ˜ difference in effects between two groups R c „ ceiling“value (maximum acceptable value) of ICER (incremental cost-effectiveness ratio)To simplify the equation: z α / 2 + z β = a σ CT 2 + σ CC 2 = b R c σ ET 2 + σ EC 2 = d Δ C ˜ − R c Δ E ˜ = c

n > a d + b c 2

The equation for ε is

ε = ∂ f ∂ b ⋅ b f
= 2 a d + b c â‹… a c â‹… b a â‹… d + b c 2
= 2 b d + b

If the standard deviation increases by 1 percent, the sample size has to increase by = 2 b d + b percent. Since b>0 and d>0, the elasticity for cost-effectiveness is lower than for a single outcome.

References

  1. Breyer F, Felder S: Life expectancy and health care expenditures: a new calculation for Germany using the costs of dying. Health Policy. 2006, 75 (2): 178-186. 10.1016/j.healthpol.2005.03.011.

    Article  PubMed  Google Scholar 

  2. Schulz E, Leidl R, Konig HH: The impact of ageing on hospital care and long-term care–the example of Germany. Health Policy. 2004, 67 (1): 57-74. 10.1016/S0168-8510(03)00083-6.

    Article  PubMed  Google Scholar 

  3. Seshamani M, Gray A: Ageing and health-care expenditure: the red herring argument revisited. Health Econ. 2004, 13 (4): 303-314. 10.1002/hec.826.

    Article  PubMed  Google Scholar 

  4. Cost-effectiveness in health and medicine. Edited by: Gold MR, Siegel JE, Russell LB, Weinstein MC. 1996, New York: Oxford University Press

    Google Scholar 

  5. Ramsey SD, McIntosh M, Sullivan SD: Design issues for conducting cost-effectiveness analyses alongside clinical trials. Annu Rev Public Health. 2001, 22: 129-141. 10.1146/annurev.publhealth.22.1.129.

    Article  CAS  PubMed  Google Scholar 

  6. Glick H, Doshi JA, Sonnad SS, Polsky D: Economic evaluation in clinical trials. Handbooks in health economic evaluation series. Edited by: Gray A, Briggs A. 2007, Oxford University Press, New York

    Google Scholar 

  7. Schulman KA, Glick H, Buxton M, Sculpher M, Backhouse M, Bell L, Eisenberg JM: The economic evaluation of the first study: design of a prospective analysis alongside a multinational phase III clinical trial. Flolan international randomized survival trial. Control Clin Trials. 1996, 17 (4): 304-315. 10.1016/0197-2456(95)00166-2.

    Article  CAS  PubMed  Google Scholar 

  8. Ridyard CH, Hughes DA: Methods for the collection of resource use data within clinical trials: a systematic review of studies funded by the UK health technology assessment program. Value in health: the journal of the International Society for Pharmacoeconomics and Outcomes Research. 2010, 13 (8): 867-872. 10.1111/j.1524-4733.2010.00788.x.

    Article  Google Scholar 

  9. Bhandari A, Wagner T: Self-reported utilization of health care services: improving measurement and accuracy. Med Care Res Rev. 2006, 63 (2): 217-235. 10.1177/1077558705285298.

    Article  PubMed  Google Scholar 

  10. Schweikert B, Hahmann H, Leidl R: Development and first assessment of a questionnaire for health care utilization and costs for cardiac patients. BMC Health Serv Res. 2008, 8: 187-10.1186/1472-6963-8-187.

    Article  PubMed  PubMed Central  Google Scholar 

  11. Johnston K, Buxton MJ, Jones DR, Fitzpatrick R: Assessing the costs of healthcare technologies in clinical trials. Health Technol Assess. 1999, 3 (6): 1-76.

    CAS  PubMed  Google Scholar 

  12. O'Sullivan AK, Thompson D, Drummond MF: Collection of health-economic data alongside clinical trials: is there a future for piggyback evaluations?. Value Health. 2005, 8 (1): 67-79. 10.1111/j.1524-4733.2005.03065.x.

    Article  PubMed  Google Scholar 

  13. Clarke PM, Fiebig DG, Gerdtham UG: Optimal recall length in survey design. J Health Econ. 2008, 27 (5): 1275-1284. 10.1016/j.jhealeco.2008.05.012.

    Article  PubMed  Google Scholar 

  14. Goossens ME, Rutten-van Molken MP, Vlaeyen JW, van der Linden SM: The cost diary: a method to measure direct and indirect costs in cost-effectiveness research. J Clin Epidemiol. 2000, 53 (7): 688-695. 10.1016/S0895-4356(99)00177-8.

    Article  CAS  PubMed  Google Scholar 

  15. Clark RE, Teague GB, Ricketts SK, Bush PW, Keller AM, Zubkoff M, Drake RE: Measuring resource use in economic evaluations: determining the social costs of mental illness. J Ment Health Adm. 1994, 21 (1): 32-41. 10.1007/BF02521343.

    Article  CAS  PubMed  Google Scholar 

  16. Heinrich S, Luppa M, Matschinger H, Angermeyer MC, Riedel-Heller SG, Konig HH: Service utilization and health-care costs in the advanced elderly. Value in health: the journal of the International Society for Pharmacoeconomics and Outcomes Research. 2008, 11 (4): 611-620. 10.1111/j.1524-4733.2007.00285.x.

    Article  Google Scholar 

  17. van Roijen LH, van Straten A, Al M, Rutten F, Donker M: Cost-utility of brief psychological treatment for depression and anxiety. Br J Psychiatry. 2006, 188: 323-329. 10.1192/bjp.188.4.323.

    Article  PubMed  Google Scholar 

  18. Kimman ML, Dirksen CD, Voogd AC, Falger P, Gijsen BC, Thuring M, Lenssen A, van der Ent F, Verkeyn J, Haekens C, et al: Economic evaluation of four follow-up strategies after curative treatment for breast cancer: results of an RCT. Eur J Cancer. 2011, 47 (8): 1175-1185. 10.1016/j.ejca.2010.12.017.

    Article  CAS  PubMed  Google Scholar 

  19. Kirchberger I, Meisinger C, Seidl H, Wende R, Kuch B, Holle R: Nurse-based case management for aged patients with myocardial infarction: study protocol of a randomized controlled trial. BMC Geriatr. 2010, 10: 29-10.1186/1471-2318-10-29.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Tan SS, Rutten FF, van Ineveld BM, Redekop WK, Hakkaart-van Roijen L: Comparing methodologies for the cost estimation of hospital services. The European journal of health economics: HEPAC: health economics in prevention and care. 2009, 10 (1): 39-45.

    Article  CAS  PubMed  Google Scholar 

  21. Coyle D, Lee KM: The problem of protocol driven costs in pharmacoeconomic analysis. Pharmacoeconomics. 1998, 14 (4): 357-363. 10.2165/00019053-199814040-00003.

    Article  CAS  PubMed  Google Scholar 

  22. Mühlberger N, Behrend C, Stark R, Holle R: Database-supported identification and entry of drug data in health studies - experience with the IDOM software [datenbankgestützte online-erfassung von arzneimitteln im rahmen gesundheitswissenschaftlicher studien erfahrungen mit der IDOM-software] in German. Informatik Biometrie und Epidemiologie in Medizin und Biologie. 2003, 34 (4): 601-611.

    Google Scholar 

  23. Sozialgesetzbuch (SGB), Elftes Buch (XI), Soziale Pflegeversicherung. http://www.sozialgesetzbuch-sgb.de/sgbxi/1.html Accessed April 15 2011

  24. Krauth C, Hessel F, Hansmeier T, Wasem J, Seitz R, Schweikert B: Empirical standard costs for health economic evaluation in Germany – a proposal by the working group methods in health economic evaluation. Gesundheitswesen. 2005, 67 (10): 736-746. 10.1055/s-2005-858698.

    Article  CAS  PubMed  Google Scholar 

  25. Kassenärztliche Bundesvereinigung/Grunddaten zur vertragsärztlichen Versorgung. 2009, Available from URL: http://daris.kbv.de/daris.asp [Accessed 11 April 2011], , Tabelle II.1 S.39

  26. GKV-Heilmittel-Informationssystem. Available from URL: http://www.gkv-his.de/upload/HIS-Bericht-Bund_200804_562.pdf Accessed April 12 2011

  27. Standardtarif. Available from URL: http://www.debeka.de/service/bedingungen/Krankenversicherung/Allgemeine_Bedingungen/CKV50.pdf Accessed April 13 2011

  28. Statistical yearbook 2010 for the federal republic of germany including 'international tables'. Edited by: Office FS. 2010, Wiesbaden: Federal Statistical Office

    Google Scholar 

  29. Gesundheitsberichterstattung. Available from URL: http://www.gbe-bund.de/oowa921-install/servlet/oowa/aw92/dboowasys921.xwdevkit/xwd_init?gbe.isgbetol/xs_start_neu/&p_aid=i&p_aid=96674796&nummer=531&p_sprache=D&p_indsp=−&p_aid=17933936 [Accessed April 12 2011]

  30. Gesundheitsberichterstattung. Available from URL: http://www.gbe-bund.de/oowa921-install/servlet/oowa/aw92/dboowasys921.xwdevkit/xwd_init?gbe.isgbetol/xs_start_neu/&p_aid=i&p_aid=96674796&nummer=531&p_sprache=D&p_indsp=−&p_aid=17933936 Accessed April 12 2011

  31. Rehabilitation. Available from URL: http://www.deutsche-rentenversicherung-bund.de/cae/servlet/contentblob/89948/publicationFile/19214/statistikband_reha_2008_pdf.pdf;jsessionid=564C005D3595BEEC1AB4D126E3B6AC84.cae03 Accessed November 26 2010

  32. Wissenschaftliches Institut der AOK. Available from URL: http://www.wido.de/arzneimittel.html Accessed April 15 2011

  33. Vertrag gemäß § 89 SGB XI. Available from URL: http://www.vdak-aev.de/LVen/BAY/Vertragspartner/Pflegeversicherung/aPfVerguetung2.pdf Accessed April 15 2011

  34. Briggs AH, Gray AM: Power and sample size calculations for stochastic cost-effectiveness analysis. Medical decision making: an international journal of the Society for Medical Decision Making. 1998, 18 (2 Suppl): S81-S92.

    Article  CAS  Google Scholar 

  35. Hamm CW: Guidelines: acute coronary syndrome (ACS). II: acute coronary syndrome with ST-elevation. Z Kardiol. 2004, 93 (4): 324-341. 10.1007/s00392-004-0109-x.

    Article  CAS  PubMed  Google Scholar 

  36. Hamm CW: Guidelines: acute coronary syndrome (ACS). 1: ACS without persistent ST segment elevations. Z Kardiol. 2004, 93 (1): 72-90. 10.1007/s00392-004-1064-2.

    Article  CAS  PubMed  Google Scholar 

  37. Barber JA, Thompson SG: Analysis and interpretation of cost data in randomised controlled trials: review of published studies. BMJ. 1998, 317 (7167): 1195-1200. 10.1136/bmj.317.7167.1195.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

Pre-publication history

Download references

Acknowledgements

The KORINNA study is a subproject of KORA-Age. The KORA research platform (KORA, Cooperative Research in the Region of Augsburg) was initiated and financed by the Helmholtz Zentrum München - German Research Center for Environmental Health, which is funded by the German Federal Ministry of Education and Research and by the State of Bavaria. The KORA-Age project was financed by the German Federal Ministry of Education and Research (BMBF FKZ 01ET0713) as part of the ‘Health in old age’ programme.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Hildegard Seidl.

Additional information

Competing interests

The authors declare that they have no competing interests.

Authors’ contributions

HS determined unit prices and calculated costs, performed the statistical analyses, calculated the elasticity and drafted the manuscript. CM conceived the KORINNA study and participated in its design and coordination. RW collected the data and coordinated the KORINNA study. RH conceived the KORINNA study, participated in its design and coordination and helped to draft the manuscript. All authors read and approved the final manuscript.

Authors’ original submitted files for images

Below are the links to the authors’ original submitted files for images.

Authors’ original file for figure 1

Rights and permissions

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article

Seidl, H., Meisinger, C., Wende, R. et al. Empirical analysis shows reduced cost data collection may be an efficient method in economic clinical trials. BMC Health Serv Res 12, 318 (2012). https://doi.org/10.1186/1472-6963-12-318

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/1472-6963-12-318

Keywords