A standardized framework to evaluate the quality of studies using TDABC in healthcare: the TDABC in Healthcare Consortium Consensus Statement

Abstract

Background

This Consensus Statement introduces a standardized framework, in a checklist format, to support future development and reporting of TDABC studies in healthcare, and to encourage their reproducibility. Additionally, it establishes the first formal networking of TDABC researchers through the creation of the TDABC in Healthcare Consortium.

Methods

A consensus group of researchers reviewed the most relevant TDABC studies available in Medline and Scopus databases to identify the initial elements of the checklist. Using a Focus Group process, each element received a recommendation regarding where in the scientific article section it should be placed and whether the element was required or suggested. A questionnaire was circulated with expert researchers in the field to provide additional recommendations regarding the content of the checklist and the strength of recommendation for each included element.

Results

The TDABC standardized framework includes 32 elements, provides recommendations where in the scientific article to include each element, and comments on the strength of each recommendation. All 32 elements were validated, with 21 elements classified as mandatory and 11 as suggested but not mandatory.

Conclusions

This is the first standardized framework to support the development and reporting of TDABC research in healthcare and to stablish a community of experts in TDABC methodology. We expect that it can contribute to scale strategies that would result in cost-savings outcomes and in value-oriented strategies that can be adopted in healthcare systems and institutions.

Background

The use of Time-Driven Activity-Based Costing (TDABC) methodology to evaluate costs in healthcare has expanded in the last 5 years, with studies encompassing different healthcare fields such as surgery and perioperative care [1, 2], cancer [3], transplants [4, 5], outpatient care [6], psychiatry [7], and hospital management [8] as reported in recent systematic reviews [9, 10]. The increasing number of TDABC studies suggests that this is an essential method for supporting value-based healthcare management programs [11,12,13]. In fact, in 2020 the Society for Perioperative Assessment and Quality Improvement (SPAQI) outlined eight principles of TDABC and argued that the aforementioned method provides cost transparency to demonstrate value in perioperative care [14]. Even though the use of TDABC has advanced, its application in microcosting studies in healthcare presents methodological heterogeneity and demonstrates differences in how researchers design studies and report their methods and results [9, 15].

The International Society for Pharmacoeconomic and Outcomes Research (ISPOR) recently revised the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement, which provides recommendations to optimize the reporting of health economic evaluations [16]. The CHEERS statement provides a list of 24 items that a scientific article on economic evaluation should contain. However, specific items in the list that guide cost reporting, i.e. “estimating resources and costs (13a and 13b)” and “Incremental costs and outcomes (19)” lack detailed methods to achieve high quality and real individual cost information. The use of bottom-up microcosting techniques are described in the literature as the “gold-standard” recommendation to evaluate the cost information in economic analyses [17, 18], and the TDABC methodology can be used to better perform the microcosting studies [9, 15].

This Consensus Statement aims to provide a standardized framework, in a checklist format, to support future development and reporting of TDABC studies in healthcare. This checklist is not intended to be a rigid guideline, but rather as a tool to assist researchers when designing and reporting TDABC studies. It should prompt reviewers and readers to question the lack or appropriateness of certain elements when developing a healthcare-related TDABC project.

Our overall goal is to encourage reproducibility in TDABC studies, while taking into account the different variables that may influence cost analysis and the different ways in which TDABC has been implemented in microcosting studies around the world. In addition, this Consensus Statement aims to establish a global community of TDABC researchers through the creation of the TDABC for Healthare Consortium (www.tdabcconsortium.com).

Methods

The overall methodology was based, at least in part, on other recent checklists used for reporting study results [19]. Our approach consisted of four phases: (i) Selection of TDABC expert researchers and identification of initial elements of the checklist; (ii) Application of a Focus Group process to build the checklist [15]; (iii) Selection of external opinions to validate the checklist; (iv) Creation of the checklist. Each phase involved different researchers who were categorized into the following groups: leaders of the consensus group (3 researchers: A.P.E., C.A.P., and R.D.U.); the consensus group (3 leaders and two additional researchers); and the TDABC consulting researchers (17 researchers invited to answer the questionnaire). Therefore, during the checklist development process, a total of 22 experts including researchers, clinicians and healthcare managers were involved. Figure 1 is a flowchart which details the sequence of consensus phases, the types of participants who were involved in each phase, techniques applied, and the results following each phase. Each of the four phases is described in more detail below.

Fig. 1

Sequence of methodological steps followed to develop a standardized framework consensus to evaluate the quality of studies using TDABC in healthcare

Selection of TDABC expert researchers and identification of initial elements of the checklist

The authors of this article represent the leaders of the consensus group, responsible for the design and management of the checklist. The TDABC methodology [19] and the 8-step framework applied in TDABC microcosting studies [4] were used as guides to create the checklist element sequence. The checklist element sequence was created by posing a question to the participant which required a binary answer (“yes” or “no”). This approach was used to help future checklist users to assess each element when reviewing a TDABC in healthcare study.

Next, the 15 most cited cost analyses or management studies that detailed opportunities in applying TDABC in healthcare, available in Scopus database, were reviewed to identify the presence of each element in the studies. The keywords “Time-Driven Activity-Based Costing” OR “TDABC” and “Health” were used to search for the articles. Studies with a high citation but that only reported the TDABC as a cost analysis strategy used while not detailing its application, were not considered (10 articles – see Additional file 1). Those articles commented on the method theoretically but did not apply it in a real-world setting. The articles used in this Consensus statement contained at least 21 citations. Each element in the articles was categorized, where “Required” was attributed to the elements encountered in all studies, and “suggested” attributed for elements encountered in only a few studies.

Since a few studies aimed to assess costs or compare the TDABC with traditional cost methods (Type I) [20,21,22] and others aimed to apply TDABC to support healthcare service management (Type II) [23,24,25], the elements were further classified as follows: (#) for elements suggested by all studies; (*) for cost assessment (Type I); and (#*) for management (Type II).

Application focus group process to build the checklist

The consensus group, formed by the project leaders and two additional researchers with significant experience in TDABC, evaluated the framework in a Focus Group section. The workshop was organized to develop and clarify the elements in the checklist and to define the best section in a research study to place each element. The group followed the sequence of reiterative steps suggested by existing Focus Group methodology for health and medical research [26]. The moderator (A.P.E.) used the researchers’ current experiences from different projects that are using TDABC to evaluate hospital costs of the bone marrow transplant treatment [4] and a telemedicine service [27] in Brazil, to facilitate the review and discussion of each element.

For the Focus Group workshop, first the moderator explained the goals of the checklist and presented the proposed elements. Next, each researcher was invited to: classify the level of importance of each element using a 5-point scale for rating [28] (1- it is not important and 5 – it is extremely important); identify if the element was important for all studies, or specific to Type I (studies that applied the TDABC only to assess costs) or Type II (studies that applied the TDABC to support healthcare service management); and identify the most appropriate section in the publication to place each element. Researchers received 1 h to individually complete the first stage. Subsequently, a discussion section using the experiences from the bone marrow transplant [4] and telemedicine [27] projects was carried out by a moderator who encouraged the interaction without creating bias [29]. For any disagreement among members, the researchers discussed their areas of disagreement and reached a consensus based on the examples that were encountered during the research group members’ recent experiences with TDABC studies [4, 5, 27] and the results encountered in the recent systematic review developed by the leaders of the consensus group [15].

With the results generated in the workshop section, the consensus group then discussed the framework to incorporate additional best practice elements.

Selection of external opinions to validate the checklist

The TDABC study checklist was distributed to authors of prior key TDABC studies, which were identified in our most recent systematic review [15] exploring TDABC in healthcare as well as to authors who frequently published studies using this methodology. A total of 17 published authors included researchers with Master’s and Doctoral degrees from North and South America, Europe, Oceania and Africa who collaborated by answering a Google Forms questionnaire (Appendix). The questionnaire contained five possible responses to define whether an element fell into one of the following categories: (i) Mandatory; (ii) Strongly suggested, but not mandatory; (iii) Suggested; (iv) Optional; (5) Do not include. The interviewees were invited to answer the questionnaire and consented to participate by having their opinions provided in the questionnaire to be included in the research analysis.

Creation of the checklist

The answers were combined into a spreadsheet by the consensus group for data analysis. The frequency for each selected answer was used as the criterion to define the strength of recommendation for each element, so that the strength of recommendation suggested by the majority of those interviewed regarding each element was used to define the final strength of recommendation. A final framework including 32 elements, the strength of each recommendation, and the location within the article where each element should be addressed, was used to generate the final framework version to evaluate the quality of studies using TDABC.

The researchers involved in all phases of this project represent the initial members of the newly-formed TDABC in Healthcare Consortium (www.tdabcconsotium.com). The Consortium is composed of a collaborative group of researchers and managers who are working to improve the quality of projects that apply TDABC in healthcare and disseminate methodological advances for TDABC to promote value-based healthcare (VBHC).

Results

Selection of TDABC expert researchers and identification of initial elements of the checklist

The general framework containing elements to guide the reporting of TDABC in healthcare studies includes the revised literature results used to classify the elements (Table 1). The first column depicts the TDABC steps, the second contains the elements, and the third indicates if the element is classified as “required” or “suggested”. The symbol (eg. #, *, #*) coding indicates if the element is recommended for all studies (#), only for Type I (*) or only Type II (#*).

Table 1 List of elements for reporting TDABC studies as identified by the consensus group

The results shown in Table 1 were the elements identified in all reviewed studies such as the justification for the use of TDABC, the involvement of a multidisciplinary team, the use of formal interviews and observations in situ, the representation of the patient flow at a macro level in a process map, and the connection with value in healthcare decisions. On the other hand, we identified significant heterogeneity in how the TDABC methodological steps were being followed in the studies, such as the patient flow map design, the financial data used (hospital, or external databases) and its level of detail, the inclusion of overhead costs, ways to estimate capacities, time data collection, and approaches used to report the results.

Application of focus group process to build the checklist

By applying the Focus Group process in the first round, we identified two different answers at a level of 2 point-difference and eight different answers at a level of 1 point-difference to confirm the level of importance of each element. We also determined the section within the published study where each element should be placed and if the placement was found in “All studies” (#), only Type I (*), or only Type II (#*). We reached the consensus for both of these questions after the first round. The divergencies of opinion observed during the first round regarding the level of importance of each item were discussed and, in the second round, a consensus was reached for all of them. Finally, suggestions of additional elements that should be included were discussed. The ones agreed upon by the consensus group were included in the checklist (bold letters in Tables 2 and 3). The TDABC elements, differences in opinion, the final consensus reached about level of importance, Type category and suggested paper section are shown in Table 2.

Table 2 The checklist with TDABC elements after the Delphi workshop

Table 3 The consensus statement for TDABC studies in healthcare in a checklist format

Three elements, which were reported differently by previous authors, were considered of lower level of importance (level 2) during our consensus process. In contrast, while studies reviewed on Table 1 identified elements as common, at the workshop they were considered important (level “5”) by consensus. Additionally, it is important to point out that this methods section concentrates on where the majority of elements should be addressed (16 of 32 elements).

Addition of external opinions and creation of the checklist

Based on the data obtained from the questionnaire, we confirmed the strength of the recommendation for each element in the framework that may be used to guide future TDABC in healthcare studies. Twenty-one elements were judged to be “Mandatory” and 11 elements were “Strongly Suggested but not Mandatory”. A few elements received one or two “Do not include” recommendations. However, we did not exclude the elements because of the “majorities’ opinion” criteria, which was used to define the strength of recommendation for each element. Table 3 shows the standardized TDABC framework to support the development and reporting of TDABC studies in healthcare.

All the elements that should be placed in the methods section were confirmed as mandatory, as was previously suggested by the level of importance registered in the Focus Group workshop, with only one exception for the element of currency and discount taxes. For this specific element, once the framework is applied to support multiregional or multi-period cost analyses, it is important to use the most current inflation information available on the world data bank website (https://data.worldbank.org/) to achieve accurate cost information and to compare costs. The following elements were all classified as mandatory and suggested to be placed in the methods section: patient flow map design, financial data used (hospital, or external databases) and its level of detail, the inclusion of overhead costs, the method used to estimate capacities, and the time data collection.

Discussion

The principles of Value-based Healthcare (VBHC) include strategies for alleviating the burden of high expenses in healthcare [13, 30, 36]. However, for successful implementation of VBHC interventions, it is important to achieve accuracy on cost information [31]. This Consensus Statement is the first to introduce a standardized framework, in a checklist format, to support the development and reporting of TDABC studies in healthcare.

The use of advanced costing methods in healthcare offers an opportunity for researchers to fully explore alternatives to improve allocation of public healthcare resources [10]. The group’s recent systematic review about TDABC in the VBHC context identified 17 studies that used the TDABC to increase value in healthcare by assessing cost-savings throughout the optimization of the episode of care-cycle [15]. Even though the TDABC was already defined as the “gold standard” to assess real costs in healthcare, the application of TDABC in scientific studies showed enormous variability on how the methodology has been employed [9, 15]. To solve this problem, we developed the framework that consolidated methodological guidance for the application of TDABC. By following the framework, it is expected that those who design the studies and interpret the results can achieve better accuracy on cost information, understand limitations of some of these studies, and identify opportunities to reduce waste along the episodes of care, all contributing to the expansion of VBHC programs that include more precise cost outcomes assessment.

Neglecting certain costs or adopting inaccurate methodologies for cost estimations can introduce bias in interpretation of the results in health economic studies [32]. As a result of our Focus Group process, it is important to note the significance attributed to the proposed elements described in the methods section. The elements suggested to be posted in the given research study method section were classified as “Mandatory” and only one as “Strongly suggested, but not mandatory”. The researchers involved in the workshop process discussed the importance of detailed reporting of the essential elements to make the TDABC results valuable and replicable for future economic analyses, with the understanding that valid value-based comparisons are not possible without consensus around how costs were being calculated [32]. The strength of the recommendations regarding the methods elements based on the questionnaire responses highlights the importance of detailing of each step followed when applying the TDABC methodology. This guarantees the rigor of the TDABC use to assess costs (Type I) or to support healthcare service management (Type II).

One important issue when applying advanced cost methods is to define the scope, objective and business context where the study is being applied [33]. If the study is specific to assessing costs, the requirements can be more focused on financial aspects; on the other hand, once the method is being used to support healthcare service management, the results and discussion may explore the cost-savings and efficiency increase opportunities [37]. The identification of elements that are recommended for all TDABC studies (Type I and Type II) and others specific for Type I or Type II studies is the approach that we suggested in the tool to guide future authors to better consolidate results from their TDABC projects, considering the management and cost assessment or only the cost assessment objective.

In all studies reviewed to introduce the consensus elements, multidisciplinary teams were used to implement the TDABC, which suggests that the individual economic and financial backgrounds vary among the team members. The establishment of transparent communication when dealing with several backgrounds is not an easy task; the terms and definitions usually vary among physicians, engineers, economists, and others. The existence of a framework in a checklist format contributes to clarifying the communication among all team members. Once we have a better comprehension about the sequence of steps that may be followed in a TDABC project, we believe that the methodological rigor for applying the TDABC can be easily evaluated, contributing to the homogeneity of the studies and consequently to the reproducibility of results. This Consensus statement coincided with the creation of a global network of TDABC researchers, the TDABC Consortium, which can further contribute to the global dissemination of the standards of TDABC research reporting, the research that is still mostly concentrated in the USA [9, 15].

TDABC is valuable because of its detailed reporting on the episode of care, as it identifies how patients consume resources along the process of care. Therefore, if well-constructed, this method can be used to optimize the care cycle [31], [41], resulting in cost-saving outcomes [22]. If authors apply their TDABC strategies with higher methodological rigor, the results from parallel studies can be compared and replicated, opening new avenues to redesign initiatives to optimize care cycles in different healthcare settings [22]. We believe that by the achieving homogeneity and accuracy in studies that applied TDABC in healthcare, the cost information can be more efficiently used for health economic analysis to guide decisions about issues such as the incorporation of health technologies, economic evaluation of healthcare cycle redesign, healthcare system management, reimbursement policies evaluation, and the identification of waste reduction opportunities in clinical pathways.

The use of TDABC to identify benchmarks of care pathways is a valuable application of the method in the context of VBHC [15]. However, once the method is being used to evaluate or compare costs and care pathways internationally, authors must take into consideration the study’s economic assessment perspective and each healthcare system’s context before accepting conclusions about costs and efficiencies based on system’s results. Recently, a TDABC study compared the potential cost-savings in implementing what were judged to be cost-effective coronary artery bypass graft surgeries performed by top Indian hospitals in contrast to USA hospitals. The study concluded that the implementation of best practices observed in Indian hospitals would require modifications in regulations, family participation in the care process, and stronger support by hospital executives and clinicians [35].

At this time, the standardized framework for TDABC is supported by the literature and by the opinions of specialists with experience in applying TDABC in healthcare, although there is a lack of real-world full applications of the frameworks, and is therefore a research limitation. Our selection of experts was based on previously published TDABC studies, but we understand that with the application of the framework, methodological improvements from additional studies can be added to the framework. Also, the framework was designed in the context of healthcare business environment, even though the TDABC method was initially developed to increase the accuracy of unitary cost information in companies with high indirect costs in a variety of business sectors [37]. Once our standardized reporting framework establishes standardized reporting of healthcare studies, a similar process can be used to adapt it to any business sector.

Regarding the TDABC in Healthcare Consortium that has helped organize this work, this consensus statement helps introduce this innovative and global network. We believe that there are future opportunities for the development of international collaborative research and conferences for knowledge sharing to consolidate scientific advances for TDABC in the VBHC context.

Conclusion

In conclusion, this Consensus Statement introduces a standardized framework, in a checklist format, for the application of TDABC to encourage researchers to perform higher quality TDABC costing studies that can be replicated, contributing to scaling of strategies that result in cost-savings in healthcare and other fields. Additionally, we believe that the standardized framework for healthcare and the creation of the TDABC Consortium can help connect and guide researchers, administrators, and other key decision-makers on how to quantitatively evaluate the impact of each strategy in projects designed to increase value in healthcare.

Future research should evaluate the benefits of the use of the TDABC standardized framework to report, in a scientific forum, TDABC projects developed in the context of healthcare business environment. This would help ensure the quality and interpretation of subsequent TDABC studies and help generate accurate cost information to support future decision-making processes.

Appendix

https://forms.gle/b38VGT8Frm4ED7yCA