Skip to main content

Costs and effectiveness of a culturally tailored communication training program to increase cultural competence among multi-disciplinary care management teams

Abstract

Background

Several studies have demonstrated that cultural competence improves patient-provider communication, which promotes adherence to established care plans and improves patient satisfaction and health outcomes. However, there is very little data available regarding the costs associated with the development and implementation of cultural competence training, or the cost-effectiveness of these programs. To that end, this evaluation aims to describe costs, program effectiveness, and cost-effectiveness of a culturally tailored communication training program to improve cultural competence among multi-disciplinary care management teams.

Methods

As part of a region-wide quality improvement initiative to reduce healthcare disparities among African American patients with uncontrolled hypertension, three multi-disciplinary care management teams were invited to participate in a two-part communication training program. A paired samples t-test was used to assess program effectiveness based on participant responses to a validated cultural competence self-assessment survey 2 weeks before and after the training program. A micro-costing approach was used to estimate programmatic costs for content development and delivery. Cost-effectiveness was then determined using the average cost-effectiveness ratio, and sensitivity analyses were conducted to assess the impact of participant mix on this result.

Results

All scores (n = 17) improved after training; however, only the culturally competent behaviors (CCB) subscale change was statistically significant (p = 0.02). Overall program costs were $5754.19. The average program cost per participant was $138.51, with an ACER of $337.83 per 1-unit increase in CCB score. Sensitivity analyses yielded a range of ACERs between $122.59 and $457.07, where all participants are support staff or nurses, respectively.

Conclusions

Culturally tailored communication training increases how frequently participants demonstrate culturally competent behaviors and may be a cost-effective intervention for care management teams to address individual cultural competence. Detailed costs associated with cultural competence training are largely unavailable in the literature; as such, these data may serve as a financial framework for organizations considering the implementation of similar programs.

Peer Review reports

Background

In 2002, the Institute of Medicine released a groundbreaking report, Unequal Treatment: Confronting Racial and Ethnic Disparities in Health Care, that concluded racial and ethnic minorities in the United States receive lower quality of care and have higher rates of morbidity and mortality from chronic diseases than their white counterparts [1]. Nearly 20 years later, racial and ethnic disparities in healthcare access and quality of care have improved; yet, for nearly 19% of access measures and 75% of quality measures, the gap between Blacks and Whites persists today [2]. Moreover, the cost of racial disparities are significant, with estimated direct healthcare costs reaching $230 billion and indirect costs by more than $1 trillion [3].

Within the same groundbreaking report, healthcare provider training was identified as one of the most important tools to overcome disparities [1]. More often than not, the purpose of this training is to improve provider cultural competence. The concept of cultural competence first emerged in the late 1970s as cross-cultural healthcare and later, in the 1990s, expanded to include all racial and ethnic minority populations across the health system [4]. Many definitions of cultural competence exist, though most agree that culturally competent care respects the diversity of the patient and meets his/her social, cultural, and linguistic needs [5].

Several studies have tested the effectiveness of cultural competence training on intermediate outcomes, such as provider knowledge, attitudes, and behaviors, and long-term outcomes, like improvement in clinical measures; however, very few studies have reported detailed costs of cultural competence training [6, 7]. In one study where costs to train physicians were included, the overall cost for the course, excluding physician time, was provided but no cost-effectiveness analysis was conducted [8]. The current analysis goes beyond earlier evaluations to both describe costs associated with training and evaluate the cost-effectiveness of such a program aimed to improve cultural competence among multi-disciplinary care management teams.

Methods

Program design

As part of a region-wide quality improvement initiative to reduce healthcare disparities among African American patients with uncontrolled hypertension, a two-part culturally tailored communication training program was developed and delivered to care management teams.

Participants

Three care management teams, comprised of 14 registered nurses, 15 registered pharmacists, and 7 non-clinical support staff, were invited to participate in the training program and self-assessment. Prior to the initiation of the program, all participants were informed that their participation in the program and assessment was voluntary.

Training program

The training was adapted from a series of resources and similar quality improvement initiative developed by, and with, Kaiser Permanente’s Equitable Care Health Outcomes team that incorporated an evidence-based communication model (AIDET – Acknowledge, Introduce, Duration, Explanation, and Thank You) and general awareness of African American culture and barriers within the healthcare system [9]. The newly developed training program consisted of a 25-min on-demand web-based module and 1.5 h live interactive session. The web-based module, “Touching the Dream: Focus on African American Culture and Health”, is part of the Diversity & Health Series created in the Kaiser Permanente Colorado Region to raise cultural awareness of healthcare providers to eliminate healthcare disparities within African American communities. The live interactive session included the delivery of a presentation, “Building Connections Using Culturally Tailored Communication: AIDET Training for African American/Black Patients”, and participant discussion of scenarios and experiences.

Effectiveness outcomes

The primary effectiveness outcome was estimated using a paired samples t-test based on the change in self-assessed cultural competency from baseline on one index and two scales within the Cultural Competence Assessment (CCA) instrument before and after the training program. The CCA was developed to assess cultural competency in a multi-disciplinary setting, with adequate test-retest validity overall (r = 0.85, p = 0.002) and for each of the subscales, culturally competent behaviors (CCB, r = 0.87, p = 0.002;) and cultural awareness and sensitivity (CAS, r = 0.82, p = 0.002) and acceptable reliability (Cronbach’s alpha = 0.89) [10, 11]. The instrument was used with permission from its creators. The index item assesses overall self-reported cultural competence using a single 5-point Likert-like question, “Overall, how competent do you feel working with people who are from other cultures different from your own?” The CAS subscale uses a 7-point Likert-like response set ranging from “strongly agree” to “strongly disagree”, while the CCB subscale uses a similar 7-point scale with responses from ‘always’ to ‘never’. Responses are translated to numerical values and summed to yield a score for the CCA index and both the CAS and CCB subscales, where higher scores reflect greater self-reported overall competence, greater knowledge, more positive attitudes, and more frequently demonstrated behaviors.

Program costs

Program costs were calculated using a micro-costing approach, where resource units are multiplied by unit costs. All participants were employees of Kaiser Foundation Health Plan at the time of training. Administrative time to develop and deliver the intervention was tracked throughout the program. Participant salaries and fringes were estimated using the most recent Bureau of Labor median occupation wages for the Atlanta-Sandy Springs-Roswell, GA metropolitan service area for the state of Georgia and employer costs for employee compensation, respectively [12, 13]. For those participants that were not located at the regional office where the training took place, travel time and distance for participants were estimated based on average time and distance from medical offices to the regional office and mileage reimbursement was calculated based on the January 1, 2019 standard mileage rate as dictated by the Internal Revenue Service [14].

Cost-effectiveness analysis

The cost-effectiveness analysis was from the employer perspective. Mean intervention costs (the costs associated with the training program) and mean effectiveness gained (the difference between pre- and post-training CCA scores, where statistically significantly different) were used to calculate the average cost-effectiveness ratio (ACER)(Eq.1). The ACER was used for this analysis because it is often used by organizations as a summary parameter that characterizes an intervention independent of a comparator, which this new program lacked, beyond the status quo, to systematically allocate budgetary resources [15, 16].

$$ ACER=\frac{ProgramDeliveryCost}{AverageScoreAfterTraining- AverageScoreBeforeTraining} $$
(1)

Sensitivity analysis was applied to establish how the costs and cost-effectiveness of the program might vary from the base-case for three participants mix scenarios: all nurses, all pharmacists, or all support staff.

Institutional Review Board review was not sought for this work, as the authors determined the project did not meet the Federal definition for human subjects research, instead, falling within the scope of quality improvement. Quality improvement initiatives are exempt from Institutional Review Board review, by definition, as these initiatives do not aim to create new knowledge, per se, but rather use existing knowledge to improve health outcomes within a local system [17, 18].

Results

Of those invited to participate in the training program, 92% (n = 34) completed the web-based module and attended the group session. There were no observed differences between respondents and non-respondents.

Effectiveness

Survey responses were received from 100% (n = 36) of participants before the training and 61% (n = 22) after the training. Of the participants that completed the self-assessment before and after the training, 47% (n = 17) disclosed enough information to match responses for inferential analysis (Table 1).

Table 1 Mean Cultural Competence Assessment (CCA) Scores (Matched) Before and After Training (n = 17)

There was no statistically significant difference between overall cultural competence scores before (M = 4.59; SD = 0.51) and after training (M = 4.76; SD = 0.43); t (16)=1.74; p = 0.09. Similarly, no statistically significant difference was observed between CAS scores before (M = 6.03; SD = 0.41) and after training (M = 6.04; SD = 0.48); t (16)=1.74; 0.46. However, there was a statistically significant difference between CCB scores before (M = 4.31; SD = 0.90) and after training (M = 4.72; SD = 0.93); t (16)=1.74; p = 0.02.

Costs

Costs were divided into two categories: program development and program delivery. The costs to develop the training program, $1044.98, were predominately administrative while costs to deliver the program, $4709.21, were largely attributed to participant time (Table 2).

Table 2 Total Costs to Develop and Deliver the Culturally Tailored Communication Training Program

Cost-effectiveness

Base-case and sensitivity analysis costs and ACERs are reported in Table 3. The base-case ACER was $337.83 per 1-unit increase in the frequency with which care management teams demonstrate culturally competent behavior. The sensitivity analyses yielded a range of ACERs for a 1-unit change in CCB performance between $122.59 when all participants are support staff to $457.07 when all participants are nurses.

Table 3 Base-case and Sensitivity Analysis Intervention Costs and Cost-effectiveness Ratiosa

Discussion

Culturally tailored communication training significantly improved the frequency with which members of care management teams within an integrated delivery network reported demonstrating culturally competent behaviors. This finding is consistent with those of other larger, more robust interventions that reported beneficial effects of cultural competence training on healthcare providers’ skills and behaviors [6]. Of the three dimensions assessed within the cultural competence assessment, only culturally competent behaviors significantly increased, likely because the training program provided communication tools that could be implemented in every day practice. This finding is encouraging as communication is one behavior which cultural competence training programs aim to improve among healthcare providers and teams.

No significant improvement in overall cultural competence or cultural awareness and sensitivity was found after our training program. Participants’ self-assessments on each of these cultural competence measures was high prior to the training, which may be explained in that 75% of survey respondents indicated they had received prior cultural competence training. This explanation is consistent with the contrasted groups assessment within the CCA psychometric evaluation study, where scores for those that reported receiving training previously were significantly higher than those without prior training [11]. Moreover, Kaiser Permanente has long been recognized for its organizational climate that fosters cultural competence, which a recent study found has a direct impact on providers’ confidence in their own cultural competence [19].

The total cost to develop and deliver the training was approximately $5800; however, nearly 20% reflects a one-time cost to the organization for the development of the initial program. It is possible that these costs may underestimate the costs to other organizations for the development of a similar training program as there were no costs incurred for the “Touching the Dream” video, initial training content (which was modified), and rental space. Even so, the Office of Minority Health offers many educational resources at no charge through its website, Think Cultural Health, which organizations may use to improve cultural competence across many different healthcare roles [20]. To minimize the disclosure of sensitive financial information, local average wages and fringes from labor bureau statistics were used in this analysis, so organizations may choose to compare these data to actual salaries and wages before moving forward with developing their own cultural competence training program.

Because this was a new program and there were no active programs to which a comparison could be made, the cost-effectiveness analysis used for this program evaluation was the ACER. The use of an ACER to report the results of a CEA are likely less well known to the reader than the use of incremental cost effectiveness ratios (ICERs). However, well-known organizations have recommended the use of ACERs for similar purpose and the evaluation of de novo interventions, e.g. the World Health Organization recommends using ACERs for programs where resource allocation is limited by budgetary constraints and the Centers for Disease Control and Prevention has published an ACER-based CEA of West Nile vaccination [16, 21].

The average cost of the program per participant was $138.51. The ACER of the training program was $337.83 per 1-unit increase in program effectiveness. Program effectiveness used in the calculation of the ACER was based solely on the culturally competent behavior (CCB) subscale, as this was the only statistically significant improvement observed in our analysis. Most cost-effectiveness analyses (CEAs) use quality-adjusted life years (QALYs) with commonly agreed upon thresholds upon which cost-effectiveness is compared. There are few thresholds available for CEAs based on natural units; instead, the costs of the program are often compared to the costs of an occurred or avoided event. There is no readily available data regarding healthcare costs associated with the frequency of culturally competent behaviors which leaves organizations to subjectively assess the worth of increasing the frequency of nurses, pharmacists, and non-clinical support staff from “often” to “somewhat often” or “very often” to “always”.

When participant role and effectiveness by role was altered, the ACERs ranged from $122.59 (support staff only) to $457.07 (nurses only). These ACERs underscore the impact of balancing financial investment in the training program and the expected outcome of the program. For instance, pharmacists reported the largest improvement from baseline (0.83) yet their salaries tend to be roughly 65% higher than nurses and nearly triple that of support staff, which yielded an ACER one-half that of the nurses-only scenario. The lowest ACER, $122.59, represented the support-staff only scenario and may present the greatest return on the organization’s investment as some case studies have demonstrated the role that these frontline individuals can play in improved clinical and financial outcomes [22].

There are several limitations of this evaluation. The small sample size significantly limits our ability to confidently demonstrate an effect on cultural competence because of the training. However, it is likely that other programs would implement this type of training in groups of similar size in accordance with standards set forth by accrediting bodies like The Joint Commission or professional associations like the American Nurses Association [23, 24]. Therefore, the value of this analysis may lie more in the micro-costing data than the methodological limitations of the effectiveness analysis. While we accounted for the lack of an active comparator by using an appropriate CEA method, future research should seek to evaluate the differences between various training programs that would likely result through iterations of quality improvement work within an organization. We acknowledge that, given the organizational culture, employees’ self-assessed cultural competence may not represent that of other organizations; employees or those in other healthcare settings. There is no information on sustained changes in cultural competence as changes in self-reported cultural competence, cultural awareness and sensitivity, and culturally competent behaviors were only assessed at one point in time, 2 weeks following the initial training. Participants’ responses may have been biased by concerns about social desirability or perceived social norms. Lastly, program effectiveness was limited to participant self-report and did not consider other outcomes like patient experience or improvement in clinical parameters.

Conclusions

Despite these limitations, we believe this paper provides substantive contributions to the vast body of cultural competence training outcomes. The outcome of the program is congruent with other programs that aim to improve skills and behaviors. Moreover, this is the first cultural competence training evaluation that includes micro-costing and cost-effectiveness outcomes, which may serve as a financial framework for other organizations considering the implementation of a similar program.

Availability of data and materials

The datasets generated and/or analyzed for this evaluation are not publicly available due to the sensitive nature of employee and employer data.

Abbreviations

ACER:

Average cost-effectiveness ration

AIDET:

Acknowledge, Introduce, Duration, Explain, Thank

CAS:

Cultural awareness and sensitivity

CCA:

Cultural competence assessment

CCB:

Culturally competent behaviors

CEA:

Cost-effectiveness analysis

QALY:

Quality-adjusted life years

References

  1. 1.

    Institute of Medicine. In: Smedley BD, Stith AY, Nelson AR, editors. Unequal Treatment: Confronting Racial and Ethnic Disparities in Health Care; 2003. https://doi.org/10.17226/10260.

    Google Scholar 

  2. 2.

    National Healthcare Quality and Disparities Report. AHRQ--Agency for Healthcare Research and Quality: Advancing Excellence in Health Care. http://www.ahrq.gov/research/findings/nhqrdr/nhqdr17/index.html. Published July 3, 2018. Accessed 26 Feb 2019.

  3. 3.

    Laveist TA, Gaskin D, Richard P. Estimating the economic burden of racial health inequalities in the United States. Int J Health Serv. 2011;41(2):231–8. https://doi.org/10.2190/hs.41.2.c.

    Article  PubMed  Google Scholar 

  4. 4.

    Saha S, Beach MC, Cooper LA. Patient centeredness, cultural competence and healthcare quality. J Natl Med Assoc. 2008;100(11):1275–85. https://doi.org/10.1016/s0027-9684(15)31505-4.

    Article  PubMed  PubMed Central  Google Scholar 

  5. 5.

    Betancourt JR. Defining cultural competence: a practical framework for addressing racial/ethnic disparities in health and health care. Public Health Rep. 2003;118(4):293–302. https://doi.org/10.1093/phr/118.4.293.

    Article  PubMed  PubMed Central  Google Scholar 

  6. 6.

    Beach MC, Price EG, Gary TL, et al. Cultural competency: a systematic review of health care provider educational interventions. Med Care. 2005;43(4):356–73. https://doi.org/10.1097/01.mlr.0000156861.58905.96.

    Article  PubMed  PubMed Central  Google Scholar 

  7. 7.

    Truong M, Paradies Y, Priest N. Interventions to improve cultural competency in healthcare: a systematic review of reviews. BMC Health Serv Res. 2014;14(1). https://doi.org/10.1186/1472-6963-14-99.

  8. 8.

    Mazor SS, Hampers LC, Chande VT, Krug SE. Teaching Spanish to pediatric emergency physicians. Arch Pediatr Adolesc Med. 2002;156(7):693. https://doi.org/10.1001/archpedi.156.7.693.

    Article  PubMed  Google Scholar 

  9. 9.

    Bartolome RE, Chen A, Handler J, Platt ST, Gould B. Population care management and team-based approach to reduce racial disparities among African Americans/blacks with hypertension. Perm J. 2015;20(1):53–9. https://doi.org/10.7812/tpp/15-052.

    Article  Google Scholar 

  10. 10.

    Schim SM, Doorenbos AZ, Miller J, Benkert R. Development of a cultural competence assessment instrument. J Nurs Meas. 2003;11(1):29–40. https://doi.org/10.1891/106137403780954949.

    Article  PubMed  Google Scholar 

  11. 11.

    Doorenbos AZ, Schim SM, Benkert R, Borse NN. Psychometric evaluation of the cultural competence assessment instrument among healthcare providers. Nurs Res. 2005;54(5):324–31.

    Article  Google Scholar 

  12. 12.

    US Bureau of Labor Statistics. May 2017 State occupational employment and wage estimates Georgia. Occupational Employment Statistics. https://wwwblsgov/oes/current/oes_12060htm#29-0000 Published March 30, 2018. Accessed 25 Feb 2019.

  13. 13.

    US Bureau of Labor Statistics. Table 14. Private industry, health care and social assistance workers, by industry and occupational group. Employer Costs for Employee Compensation. https://www.bls.gov/news.release/ecec.t14.htm. Published December 14, 2018. Accessed 26 Feb 2019.

  14. 14.

    Internal Revenue Service. IRS issues standard mileage rates for 2019. https://www.irs.gov/newsroom/irs-issues-standard-mileage-rates-for-2019. Published December 14, 2018. Accessed 26 Feb 2019.

  15. 15.

    Laska EM, Meisner M, Siegel C. The usefulness of average cost-effective ratios. Health Econ. 1997;6:497–504.

    CAS  Article  Google Scholar 

  16. 16.

    Murray CJ, Evans DB, Acharya A, Baltussen RM. Development of WHO guidelines on generalised cost-effectiveness analysis. Geneva: Global Programme on Evidence for Health Policy, WHO; 1999.

    Google Scholar 

  17. 17.

    Institute of Medicine. Crossing the quality chasm: a new health system for the 21st century. Washington, DC: National Academies Press; 2001.

    Google Scholar 

  18. 18.

    National Strategy for Quality Improvement in Health Care. Annual Progress Report to Congress. Washington, DC: U.S. Department of Health and Human Services; 2012.

    Google Scholar 

  19. 19.

    Purnell TS, Marshall JK, Olorundare I, et al. Provider perceptions of the organizations cultural competence climate and their skills and behaviors targeting patient-centered Care for Socially at-Risk Populations. J Health Care Poor Underserved. 2018;29(1):481–96. https://doi.org/10.1353/hpu.2018.0032.

    Article  PubMed  Google Scholar 

  20. 20.

    Office of Minority Health. Education. Think Cultural Health. https://www.thinkculturalhealth.hhs.gov/education. Accessed 28 Feb 2019.

  21. 21.

    Zohrabian, A., Hayes, E. B., & Petersen, L. R. (2006). Cost-effectiveness of West Nile virus vaccination. Emerg Infect Dis, 12(3), 375–380. https://dx.doi.org/https://doi.org/10.3201/eid1203.050782.

  22. 22.

    West M, Patel K, Nadel J, West M, Patel K, Nadel J. Redesigning the Care Team: The Critical Role of Frontline Workers and Models for Success. brookings.edu. https://www.brookings.edu/research/redesigning-the-care-team-the-critical-role-of-frontline-workers-and-models-for-success/. Published July 28, 2016. Accessed 28 Feb 2019.

  23. 23.

    Harper MG, Maloney P. Nursing professional development: scope and standards of practice. 3rd ed. Chicago: Association for Nursing Professional Development; 2016.

    Google Scholar 

  24. 24.

    Advancing Effective Communication, Cultural Competence, and Patient- and Family-Centered Care: A Roadmap for Hospitals. Oakbrook Terrace: The Joint Commission; 2010.

Download references

Acknowledgements

Not applicable.

Funding

Not applicable.

Author information

Affiliations

Authors

Contributions

MP developed and delivered the program, completed all primary data analyses, and was the lead contributor in writing the manuscript. XF consulted in the economic analysis and was a major contributor in writing the manuscript. AB consulted in the program evaluation and was a major contributor in writing the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Marie Parker.

Ethics declarations

Ethics approval and consent to participate

Ethics approval and consent to participate were not required for this evaluation as it was deemed quality improvement. However, all participants were made aware, prior to enrollment in the training program, that participation in the program and/or assessment was voluntary. To inform participants of their voluntary participation and the intent of the assessment, the following language was provided on the first page of the survey:

This survey is designed to explore your knowledge, feelings, and actions when you interact with others in the context of health care and health service environments. Your answers are strictly confidential. The information you provide will be compiled for overall group cultural competence and educational needs, and used to design cultural competency training programs to meet specific needs. Neither your identity nor your individual answers will be shared with anyone.

Questions on this survey are intended to gather information about how you personally think, feel, and act. Some questions may not fit your situation exactly, depending on the type of work you do at this time. Please try to answer every question. If you are unsure or have no opinion on the item, use the ‘No Opinion’ or ‘Not Sure’ options. There are no ‘right’ or ‘wrong’ answers.

Completing this survey is completely voluntary. It will take about 10 min of your time. While you may choose not to participate, your participation in the completion of this survey is encourage and appreciated. You may stop at any time. Your completion of this survey indicates your informed consent to participate in this evaluation.

Consent for publication

Not applicable.

Competing interests

The authors declare they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Parker, M., Fang, X. & Bradlyn, A. Costs and effectiveness of a culturally tailored communication training program to increase cultural competence among multi-disciplinary care management teams. BMC Health Serv Res 20, 784 (2020). https://doi.org/10.1186/s12913-020-05662-z

Download citation

Keywords

  • Cultural competence
  • Costs
  • Micro-costing
  • Cost-effectiveness
\