Skip to main content

Table 2 Main findings: evaluation and outcomes

From: Cultural competence in mental health care: a review of model evaluations

Study

Evaluation

Outcomes

Ferguson (2003)

Likert ratings (1–5) of overall value, clarity of objectives, instructor effectiveness

For second cohort (2000, N = 55):

Showed high scores on all of these: means 4.1 to 4.4 for each domain, and for two year bands 1999 and 2000 :

Intention to change: M1: 5.4% (n = 55), M2: 48.1% (n = 54), M3: 30.2% (n = 43)

Actual Change in behaviour: M1: 16.6% (n = 42), M2: 21.4% (n = 42), M3: -

Hadwiger (1999)

Cultural Sensitivity (40% of course marks)

Evidence of context of own cultural background

considered

Ethnocentric attitudes

Power orientation

Egalitarian relationship

Trust in relationships

Respect for patient during hypothetical negotiations

Manner of addressing hypothetical patients

Accuracy of content (30% of course marks)

Process (30% of course marks)

Nursing students were able to become more aware of how their own culture affects the nursing care

Able to refine cultural competence skills using hypothetical cases and narrative writing

Actual marks or origins of students not given

Siegel et al (2003) †

For each level and domain, experts identified key performance indicators identified, performance measures defined, and data sources outlined.

163 indicators

231 measures

Without a formal commitment to the development of a process and the dedication of resources for this effort, cultural competence would be difficult to achieve.

Reduced to 85 measures

Administrative:

Services.53% had put into place services that had been adapted or developed for specific cultural groups.

CC Outcomes. 60% of administrative entities indicated that outcome measures could be analyzed for specific cultural groups.

CC Training and Education.

73% indicated staff members receive ongoing education and training related to CC.

87% selected, developed, and/or provided CC training materials to agencies under their purview but only one provided financial assistance to agencies under its purview for conducting CCT

Services. 87% of the service entities indicated that they had services adapted or developed for specific cultural groups. 29% of these, providing culture-specific services was the mission of the agency; while for the remaining 71%, culture-specific services had been put in place in response to the perceived needs of clients in the community.

CC Training and Education.

75% indicated that staff of receive ongoing education and training on CC. 87% said all new employees receive CC education and training as part of their orientation.

75% said that professional education (for example, grand rounds) included racial/ethnic/cultural issues.

CC Outcomes. Outcome measure data were collected inconsistently at the five agencies responding to this question, but all conducted consumer satisfaction surveys. Sixty percent of those responding indicated that the outcome measures could be analyzed for specific cultural groups. 50% said that CC was included in staff performance evaluations.

Kim-Godwin (2001)

Literature review and concept analysis lead to 3 constructs that were evaluated: 1) health care systems, 2) health outcomes, and 3) cultural competence scale ratings.

In factor analyses, cultural knowledge emerged as a components of cultural sensitivity and cultural skills

All 13 participants reported that cultural competent care resulted in positive health outcomes in their practice. Specifically, increases in prenatal visits, higher rates of immunization, reduced morbidity and mortality, increased compliance, increased trust, increased self worth, more interest in promoting health. (Actual accounts not presented, only surmises findings).

Kondrat et al (1999)

Nature of interactions between service providers and Caucasian and African American consumers with SMI

Themes: Types and locations of service delivery

Structure of delivery services

Formal and informal organisational culture

Decision making process

Perceptions of interactions, processes and decisions

Analysis based on 700 observations across four sites

Constant comparison analyses

All four agencies incorporated policies to support diversity, yet outcomes for diverse clients varied.

11 clusters of activity:

Differentiating: B > C

1. Agency work culture: pro-agency culture:

2. Openness/boundary flexibility

3. Prevalent supervisory style: consistent, pro-active, and supportive

4. Team functioning and decisions

Non-Differentiating

5. Attitudes towards clients:

6. Demonstration of programme commitment to diversity

7. Level of acceptance

8. Diversity as a clinical issue

9. Clinical orientation

10. Level of interdisciplinary work

11. Organisation of service

There was little evidence that race or culture was routinely considered in making treatment decisions

Kirmayer et al (2003)

Participant observation of the first 100 referred cases.

29 referring clinicians for 47 cases completed service evaluation information

Specialized cultural consultation services can play a major role in educating clinicians and in developing innovative intervention strategies

Cases seen by the team demonstrated the impact of cultural misunderstandings: incomplete assessments, incorrect diagnoses, inadequate or inappropriate treatments, and failed treatment alliances.

86% of clinicians referring patients to the service reported high rates of satisfaction, but many indicated a need for longer term follow up.

41%: increased knowledge of social, cultural or religious aspects of cases

21%: increased knowledge of psychiatric or psychological aspects of their cases

48% : improved treatments

31%: improved communications, empathy, understanding, therapeutic alliance

14%: increased confidence in diagnosis, treatment

Dissatisfaction with:

14%: lack of treatment or more intensive follow up

14% unavailability or inappropriateness of recommended resources

10% concerns about the cultural appropriateness of the cultural broker

10%: too much focus on social context rather than psychiatric issues

For 21 cases, some aspects of the recommendations were not implemented: patient non-compliance (13), lack of staff or resources (9), spontaneous improvement (7).

Frusti et al (2003)

Consultant employed to assess drivers, linkages, culture and measurement strengths and weaknesses of organisation

Drivers: 1) nursing diversity committee promotes supportive work environment by sponsoring educational activities & newsletter

2) Nursing recruitment and retention committee

3) Transcultural patient care committee, provides up to date resources about influence of culture on health

Linkages:1) Managers and staff share department committee responsibilities, and feed into a shared decision making process

2) Nursing and human resources departments conduct annual planning to identify shared goals, and recruitment targets national and local nursing organisations

3) Summer intern programme to recruit under represented groups

Culture: 1) education and orientation to culture of nursing department, leadership roles developed; focus groups indicated managers are trusted, 75% of participants said they were set up to succeed by their nurse mangers

2) primary values: needs of patients first, best nursing care in the world Measurement: Recruitment data, retention data, compared with national benchmarks

Stork et al

Used data from Rosenbaum (1999) study of cultural competence in manage care contracts.

Analysis of contract excerpts for cultural competence definitions, and requirements for service provision.

Open ended interviews with officials in five states to examine written cultural competence requirements.

Purposive sample of states that

1) that had contract with cultural competence provisions

2) more comprehensive requirements than other states, reflecting early implementation

3) were average in resources and populations

4) had officials who could talk in depth about contracts

Rosenbaum reported on 37 states, of these 27 had cultural competency requirements, and 10 met criteria. : contract language comprehensive, 2) specific wording about practices rights to culturally competent services

States selected because of geographic, ethnic and racial diversity

Interview: definition of CC

Contract language/standards

Methods to measure and enforce standards

Methods to track cultural competence

Methods to track consumer

enrolment/satisfaction/service use by ethnic/racial groups

Lack of indicators for cultural competence, reluctant to enforce existing standards, disagreement over costs, lack of constituency in training and tracking

4 of five states included their own definitions of cultural competence in their contracts

• Relate to client with sensitivity, understanding, respect for clients' culture

• Understanding social, linguistic, ethnic and behavioural characteristics of a community or a population and the ability to translate systematically, that knowledge into practices in the delivery of services-identify and value difference, acknowledge interactive dynamics of cultural differences, continuously expand cultural knowledge/resources, collaborate with community re provisions and delivery, commit to cross cultural training, develop policies to provide relevant, effective, programs for diverse populations

• Ability to serve individuals of all ages, ethnic groups, in a manner appropriate to their age and unique cultural background.

• A set of congruent behaviours, attitudes and practices and policies that are formed within an agency and among professionals that enable the system, agency and professionals to work respectfully, effectively, responsibly, in diverse situations. Essential elements include: valuing diversity, understanding dynamics of difference, institutionalising cultural knowledge, and adapting to and encouraging organisational diversity.

Themes identified: contract language, contract deliverables, procedures for monitoring and oversight, data collection, provider assessments

Contractual deliverables: submit a plan to include translations of written material and access to interpreters at no extra cost, legally mandated.

Oversight/agency: assign responsibility to a specific agency.

Oversight mechanisms: readiness reviews, site reviews, before roll out. Complaint tracking, consumer satisfaction surveys.

Collecting client data: three stated did not collect enrolment data, disenrollment, provider changes, service use or satisfaction by race/ethnicity. Two states can assess requests to change provider by ethnicity, and whether change requested is a result of language problems. None of the states used the cultural data on their client to indicate lack of cultural competence. No state asks clients to rate their cultural competence of provision

Penalties: None, and none are enforced.

Assessment of CC: determined by provider to MCO/MBHO documentation of training, available ;personnel, representative services as contract deliverables dictate.

US Department of Health and Human Services (HRSA)

Organisational cultural competence assessment profile assesses domains, focus areas and indicators

Domains: As in Table 1.

For each domain there are Indicators which have a) structure, b) process and c) outcome

Findings suggest that the Assessment Profile can be useful even in its current form as an organizational framework and a guide to an organization's own development of indicators and measures of cultural competence

CC must be integrated into other organisational domains of activity

Organisational values must be tackled first.

Structures, process and outcomes agreed for each of the subheading:

Organisational Values: Leadership, investment and documentation, Information and data retrieval for cultural competence, Organisational flexibility, Community Involvement and Accountability, Board Development, Policies

Planning, monitoring, evaluation: Client, community and staff inputs, Plans and Implementation, Collection and use of cultural competence data

Communication: Understanding communication needs of clients, Culturally competent oral communication/written/other communication, Communication with community, Organisational communication

Staff development: Training commitment, Training content, Staff Performance

Organisational infrastructure: Financial, Staffing, Technology, Physical facility characteristics, Linkages

Services/Interventions: Client family community input, Screening/assessment/care planning, Treatment and follow up