Skip to main content
  • Research article
  • Open access
  • Published:

Formative process evaluation of a guideline-driven process for improving the cultural responsiveness of alcohol and drug treatment services



To improve Australian Aboriginal and Torres Strait Islander people’s access to, and experience of, healthcare services, including Alcohol and other Drug (AoD) treatment services, principles and frameworks have been developed to optimise cultural responsiveness. Implementing those principles in practice, however, can be difficult to achieve. This study has five aims: i) to describe a five-step process developed to operationalise improvements in culturally responsive practice in AoD services; ii) to evaluate the fidelity of implementation for this five-step process; iii) to identify barriers and enablers to implementation; iv) to assess the feasibility and acceptability of this approach; and v) to describe iterative adaptation of implementation processes based on participant feedback.


Participating services were 15 non-Aboriginal AoD services in New South Wales, Australia. Implementation records were used to assess the implementation fidelity of the project. Structured interviews with chief executive officers or senior management were conducted, and interview data were thematically analysed to identify project acceptability, and the key enablers of, and barriers to, project implementation. Quantitative descriptive analyses were performed on the post-implementation workshop survey data, and responses to the free text questions were thematically analysed.


A high level of implementation fidelity was achieved. Key enablers to improving culturally responsive practice were the timing of the introduction of the five-step process, the active interest of staff across a range of seniority and the availability of resources and staff time to identify and implement activities. Key barriers included addressing the unique needs of a range of treatment sub-groups, difficulty adapting activities to different service delivery models, limited time to implement change in this evaluation (three months) and the varied skill level across staff. The project was rated as being highly acceptable and relevant to service CEOs/managers and direct service staff, with planned changes perceived to be achievable and important. Based on CEO/management feedback after the project was implemented at the initial services, several improvements to processes were made.


The operationalisation of the five-step process developed to improve cultural responsiveness was feasible and acceptable and may be readily applicable to improving the cultural responsiveness of a wide variety of health and human services.

Peer Review reports


There is global concern about the impact of drug and alcohol use on health and wellbeing. The United Nations sustainable development goals include a target to strengthen the treatment of harmful substance use [1]. In Australia, enduring impacts of colonisation and racism have included Aboriginal and Torres Strait Islander peoples (hereafter referred to as Aboriginal) experiencing a disproportionate amount of harm from substance use [2]. To improve Aboriginal people’s access to, and experience of, healthcare services, including substance treatment programs, principles and frameworks have been developed to optimise cultural responsiveness (e.g. [3]). Cultural responsiveness is an ongoing process of adapting systems, services, and practice to fit with user preferences [4]. Although non-Aboriginal (mainstream) health services and clinicians may aim to provide health care that is culturally responsive to Aboriginal people, implementing its key principles can be difficult to achieve in practice [5].

Research to improve cultural responsiveness is in its infancy [6]. Key principles of culturally responsive practice for alcohol and other drug treatment services have been identified, including promoting client choice, facilitating community engagement and providing person-centred practice, which facilitate both best practice and an organisational culture which is inclusive and culturally responsive to all clients’ needs [7]. Nevertheless, barriers to practice improvement have also been identified at both the organisational-level, including leadership and organisational processes [8, 9] and the individual clinician-level, including a lack of knowledge and an attitudinal resistance to change [10]. A key hypothesis is that these barriers to implementing the principles of cultural responsiveness exist because there is a lack of clarity about how these principles can be operationalised [11].

Operationalising the principles of culturally responsive practice will require best-evidence implementation strategies [12]. Current evidence shows that since the provision of treatment guidelines, policies and training are ineffective on their own [13, 14], active learning processes are required to meaningfully engage key services’ staff in the process of change [15, 16]. The implementation process should be designed to engage staff in determining both what changes to make and how those changes should be implemented [16]. That flexibility is critical because there is evidence that different services will have different levels of cultural responsiveness at any one point in time [17] and because the approach to adopting more culturally responsive practice will need to be tailored to the individual circumstances of different services [16, 18,19,20,21].

This study has five aims: i) to describe a five-step process developed to operationalise improvements in culturally responsive practice; ii) to evaluate the level of implementation fidelity for this five-step process; iii) to identify barriers and enablers to implementation; iv) to assess the feasibility and acceptability of this approach; and v) to describe the process of iteratively adapting implementation processes based on participant feedback.


The project, including the evaluation, was funded by a consortium of Primary Health Networks (PHNs) in New South Wales (NSW) to establish cultural responsiveness guidelines for non-government alcohol and drug services (hereafter NGO AoD services). These guidelines are not intended to replace the provision of services from specialist Aboriginal AoD services or community-controlled healthcare services, but to provide guidance to enhance the cultural responsiveness of non-Aboriginal or mainstream services. The project was developed and implemented using the principles of community-based participatory research [22] and involved several distinct phases, the design, methodology and participants in each phase are described in Table 1.

Table 1 Description of design features and participants in each phase of the project

Data collection and analysis

Implementation records

Throughout the project, a tracking document was maintained by members of the project team (RW, JA, SF) to establish the number and timing of services completing each component of the project. The essential components of the project to be delivered at each service included: baseline audit, provision of written audit feedback, implementation workshop and action plan, and follow-up audit.

Interviews with CEO or senior management

Semi-structured telephone interviews regarding enablers of, and barriers to, implementation of practice change were conducted with CEOs/managers of participating services after the baseline and follow-up audit. An interview guide was followed, which aimed to capture interviewee perspectives of audit outcomes; priorities for the implementation workshop (in first interviews only); feedback on the auditing process; preferences for ongoing development of cultural responsiveness, and perceptions about changes arising from the project (in second interviews only). Interviews were conducted by a researcher independent of the participating services (SF) digitally recorded and transcribed verbatim. A report summarising the priorities identified by the CEO/manager for the implementation workshop was provided to the project team (RW, JA) to assist with planning the implementation workshops. Interview data were thematically deductively analysed to identify enablers, barriers, and acceptability of the project using NVivo 12 [24].

Implementation workshop feedback survey

At the end of the workshop, the project team provided participants with a link to a brief online anonymous 24 item survey including questions about the participants’ role, reasons for attending and perceptions of the audit and workshop (see Additional file 1: Supplementary Appendix page 1 for a list of items). The survey was hosted on the online survey platform REDCap and administered by the evaluation team. Survey participants included staff from both direct service (n = 25) and management roles (n = 10), together representing approximately one third of all staff employed by the programs included in the project. To differentiate between direct service staff and CEO/management perspectives, only the responses from the direct service staff are described here. Simple descriptive analyses (e.g. frequencies) were performed using Excel [25], and responses to the free text questions were thematically analysed using NVivo 12 [24].

Iterative adaptation of the project components during the project implementation

To iteratively adapt the project in response to feedback from CEOs/managers obtained via interviews, the project team reviewed responses to identify feedback or potential improvements around project processes, acceptability, and relevance.


Service CEOs/management provided informed consent for services to participate in audits and participants (CEOs, managers, and staff) completed individual informed consent prior to taking part in interviews, workshops, and/or surveys. Ethical approval was provided by the AHMRC [#1487/19] and UNSW Sydney Human Research Ethics Committees [REC/16/CIPHS/46].


A five-step process for operationalising improved cultural responsiveness

A five-step process to operationalise improvements in culturally responsive practice was devised, implemented, and evaluated using a program logic framework (see Additional file 2: Supplementary Appendix page 2 for the program logic).

Step 1: development of a best-practice guideline

A best-practice guideline that describes key elements of culturally responsive service delivery in non-Aboriginal AoD treatment services was developed [7]. The development of this guideline was led by an Aboriginal researcher with experience working with AoD services (RW). Extensive consultation was undertaken with Aboriginal community members in NSW including AoD clients/consumers, and with the project Advisory Group (which included senior Aboriginal AoD clinicians), to identify how they wanted AoD services to be delivered to Aboriginal people. Next existing guidelines detailing ways of working with Aboriginal people from health, community services, education and natural resource sectors and government departments were reviewed (n = 80) and strategies consistently reported in each of them were selected. This information was synthesised with feedback from the consultation into 6 themes and these themes operationalised into 21 action areas that describe key elements of culturally responsive service delivery in non-Aboriginal NGO AoD services. The themes and action areas are outlined in Table 2 and presented fully in the guideline document [7], available at

Table 2 Summary of guideline themes and action areas

Step 2: baseline audits of participating services

Structured baseline audits of current culturally responsive practice, relative to the best-practice guideline, were implemented in each participating service. Prior to the baseline audit, services were sent information explaining the audit process, what the audit is about, who should attend, how long it will take, information about the audit report and options for follow-up activities. Audits were conducted by two trained auditors who were external to the service, with at least one auditor who was Aboriginal. Audits were conducted in the setting where the service is delivered and took between 90 min to 2 h. Service staff attending the audits included a service or program manager or team leader and service delivery staff. Having a range of staff attending assisted in gaining explanations of both direct practice and organisational approaches to working with Aboriginal people. Where possible, the same staff attended the follow-up audit. Only information collected on the day of the audit was included in the scoring.

Step 3: audit feedback to participating services

Individualised written feedback from the audit findings were provided to each of the participating services. This was provided as a report listing all guideline action areas, a rating for each area reflecting the level of evidence observed during the audit (limited, some, good or excellent) and recommendations for potential areas where improvements could be made. An excerpt from an example baseline report is included in the Additional file 1: Supplementary Appendix (page 3).

Step 4: guideline implementation workshops with participating services

Implementation workshops were held with key staff (CEOs/managers and direct service staff) from services to explain the guideline, review the written audit feedback, set goals for improvement, and develop an action plan (to operationalise their improvement goals). Specifically, staff from each service were invited to select three priority action areas for their service (guided by the recommendations in the audit feedback) to progress over the next three months. They then made a detailed action plan to implement change, tailored to their specific service (an example action plan is included in the Additional file 1: Supplementary Appendix, page 5).

Step 5: follow-up audits of participating services

Follow-up audits of services were conducted after three months to assess change in culturally responsive practices, following the same procedure as for the baseline audits. Services were again provided with individualised written feedback, which also included some observations of any changes that had occurred. An excerpt from an example follow-up audit report is included in the Additional file 1: Supplementary Appendix (page 7).

The fidelity of the implementation of the five-step process

The five-step process was predominantly implemented as planned, with a high level of fidelity achieved for each of the five project components: a) development of guidelines; b) baseline audit of services; c) written feedback; d) staff attending implementation workshops; and e) follow-up audit of services. The guideline was published and provided to all participating services after the baseline audit. Twelve of the 15 participating services completed all four service-specific project components (b to e). One service completed (b) to (d), one completed (b) only, and one did not complete any of the project components (see Table 3). The main reasons for services not completing all the project components included staff turnover during the project or the project team was unable to contact service staff to arrange subsequent components.

Table 3 Implementation of each project component in participating services

Follow-up audits were scheduled to be completed 12 weeks after the baseline audit, however the mean time between audits was 18 weeks (range 14–28 weeks) (Table 3). Delays were predominantly due to scheduling commitments at services and staff availability. Of those with the longest delays, one service was re-scheduled due to a local bushfire (Cluster 3) and another was delayed due to unavailability over the Christmas holiday period (Cluster 5).

Interviews with CEOs/managers at baseline (n = 14) and follow-up audit (n = 12) showed that audit reports were received by all services that completed an audit and an interview. Staff and CEOs/managers were actively engaged in the project, with staff from 13 services attending implementation workshops (Table 3). Six CEOs/managers reported some uncertainty among some of their staff about the project requirements before the baseline audit (about the project’s background, expectations, scheduling, and next steps). These uncertainties were clarified during discussions at baseline audits. In the implementation workshop feedback survey, most direct service staff (23/25) reported that they were aware that the baseline audit had occurred at their service and over half (16/25) were aware of the outcome of the audit.

Enablers and barriers to implementation

Thematic analysis of interview responses identified CEO/manager perspectives on barriers and enablers to implementing cultural responsiveness activities (Table 4). Enablers included aligning the timing of the project with setting up new services, having multi-level buy-in for the project and having resources/staff time available to support project activities. Barriers included limited funding and time available to complete planned activities, challenges hiring Aboriginal or culturally responsive staff, the need to balance the needs of varied population groups, difficulty adapting activities to different service delivery models and limited time to implement change.

Table 4 Common enablers and barriers to implementing culturally responsive activities reported by CEOs’ and managers’

Feasibility of the project - acceptability and staff perceptions

Acceptability to CEOs/managers

The project was reported to be acceptable by all (100%) CEOs/managers who completed interviews after the baseline (n = 14) and follow-up audits (n = 12). Most reported that it had benefit to the services and to themselves, and that implementing cultural responsiveness activities was an important part of their work and a priority at their service. Many reported that their service had a focus on cultural responsiveness before the project began.

CEOs/managers’ perspectives of the workshop and audits

Thematic analysis of interviews with CEOs/Managers at follow-up (n = 12) identified several major themes as outlined in Table 5.

Table 5 CEOs’ and managers’ perspectives of the project

Acceptability and perceptions of direct service staff regarding the workshops

Twenty-five direct service or client-facing staff members from participating services attended implementation workshops and completed the survey. Participants agreed that the logistics of workshops were well organised (100%) and reported high satisfaction with the workshops (92% satisfied/very satisfied). Participants were also satisfied with the content and delivery of the workshops, with all (100%) agreeing that the workshops were well facilitated, the aims were clearly explained, the content was relevant and useful, and the workshop activities worked well, and 96% also agreeing that the planning tool was useful. All (100%) participants reported that they felt they had a clear plan after the workshop about how to change the cultural responsiveness of their service and most felt that they had the resources and support available to implement change (92%).

Iterative adaptation of the project components during the project implementation

Interview participants (CEOs/managers) were given the opportunity to provide suggestions for how the project could be improved in each interview. Suggestions were discussed among the project and evaluation team and the following components of the project were subsequently updated. Suggestions were mostly made by those interviewed from services audited early in the project. After these changes were made, no further substantive feedback was provided regarding improvements to the project.

  1. 1.

    Information provided to CEOs/managers at the beginning of the project emphasised the recommendation that a CEO/manager attend the audits, and clearly described the audit process to indicate that only information provided on the day of the audit would be included in the audit rating and report. This recommendation was made to ensure staff with knowledge around services’ processes/policies attended the audit, so information they provided could be included in the audit outcome report and considered during rating allocation. Due to the scheduling commitments of the project, the audit only captured information provided at the time of audit, and staff were not able to comment and provide additional information after the audit outcome report was provided to services.

  2. 2.

    The audit report was restructured and developed during the project. Reports were shortened and wording was revised. Numerical ratings were removed from reports. The updated version used the words ‘limited, some, good or excellent’, replacing the rating of 0, 1, 2 or 3. Changing the reporting around the rating system underpinning the audit system provided service staff with encouraging feedback, with the aim of motivating them to take on and implement the audit feedback.


The project successfully developed a best-practice guideline that describes key elements of culturally responsive service delivery in non-Aboriginal AoD treatment services [7] and implemented these guidelines in 15 NGO AoD services. This process was led by an Aboriginal researcher and involved extensive consultation with an Aboriginal advisory group, community members, and AoD service providers and organisations [26,27,28]. Overall a high level of implementation fidelity was achieved, with most participating services completing all components and both management and direct service staff demonstrating a high level of engagement with the process. Delays in completing follow-up audits was the only significant departure from the planned project implementation. Feedback from CEOs/managers indicated that the short time between audits was a barrier to the realization of more complex change activities. Future work using the five-step process devised for this project may benefit from extending the time between audits. This would provide more time for services to implement planned actions and for any changes implemented to have an impact on client outcomes.

The resources and processes developed through this project including the guidelines, audit tools, action plans and implementation workshops are feasible to use and highly acceptable to management and direct service staff at the participating services and the participatory research approach resulted in improvements in the delivery of the five-step process. While the guideline was initially thought to be the driving force underpinning the process, the importance of the other steps was quickly realised. The auditing of services, providing individualised feedback and assistance in creating action plans appeared to be key in helping services to improve cultural responsiveness. Direct service staff rated the project resources (feedback reports and action plans) as particularly useful tools for implementing change activities. As has been previously identified [15, 16], active learning processes in the implementation workshops appeared to be important in engaging both management and direct service staff and in operationalising the cultural responsiveness concepts into concrete activities.

An important strength of the project is the flexibility with which the guidelines can be implemented within individual services [19]. The recommendations in the audit reports personalise the guidelines for individual services and staff can then use the action plan to focus on those change activities identified as important and achievable for their individual services. This approach of intervening using a balance of standardised and flexible components is supported by existing evidence [16, 18, 19] and appears to be well suited to use in NGO AoD treatment services. The commitment to, and enthusiasm for the concept of cultural responsiveness from staff across multiple levels was identified as an enabler for change by CEOs/managers, consistent with previous research [16, 29]. The high level of participation and engagement from both direct service staff and CEOs/managers suggest that staff are actively aiming to enhance culturally responsive service delivery in the NGO AoD service setting. In particular, engagement of senior staff with detailed knowledge about services’ processes/policies related to cultural responsiveness and the capacity to decide and enact service level changes, appeared to be important in ensuring that planned activities were implemented into services.

Limitations to note include the relatively small number of self-selected services involved in the project, though they do represent a wide geographic and sociodemographic area of NSW and a variety of service types/delivery models. Self-selection bias is possible; services may have had pre-existing interest cultural responsiveness and/or some resources to dedicate to the process. Such a high level of engagement may not be observed in other services, given that organisational climate is a known enabler of change [16]. CEOs/managers selected directed service staff to participate in the project and not all staff at participating services were involved in the project, thus the responses described in this paper may not reflect the views of the wider service staff. There is limited information on the acceptability to those services that did not complete all components, however, drop out appears to be related to staff turnover and scheduling issues, rather than dissatisfaction with the project.

A significant barrier to actioning changes was that services lacked resources including funding and staff time with which to implement changes. Establishing cultural responsiveness practice as a routine continuous quality improvement (CQI) cycle in services might help to overcome this, and other barriers, to improvement [30]. Repeated CQI cycles have been shown to be an effective process for systems change in Aboriginal health services in a study examining diabetes care, with flow on improvements in key health indicators [31, 32]. Using this approach, services aim to incrementally improve practices over time, whilst tracking improvements through routinely collected data and regular auditing [32]. Routine evaluation frameworks could be seamlessly embedded into service delivery at low-cost by better use of administrative data that are already routinely collected. This approach would allow services time to seek out funding or resources or to schedule change activities in line with funding or other service changes. A longer-term follow-up audit of participating services is currently underway, including the identification and assessment of routine service delivery data, to evaluate the impact of the guidelines on cultural responsiveness. Future plans include working with services to develop pragmatic methods for embedding the process as a CQI, or at least ensuring that the process of improvement continues regardless of staff turnover or other organisational change. While the current project does not directly address the feasibility of scaling up the intervention, the resources developed could feasibly be utilised as a resource for all mainstream AoD services and the external auditing of mainstream services is potentially a service that could be delivered by an Aboriginal AoD organisation.


This project developed a cultural responsiveness guideline following the principles of community engagement, choice, and person-centred practice. The results support the use of a pragmatic and participatory evaluation approach using a standardised core intervention and flexible components that allowed different types of services to prioritise different aspects of cultural responsiveness and to implement solutions that suited their unique needs and strengths. The operationalisation of the five-step process developed to improve cultural responsiveness was feasible and acceptable and is readily applicable to improving the cultural responsiveness of a wide variety of health and human services.

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.



Aboriginal Drug and Alcohol Network


Aboriginal Health and Medical Research Council of NSW


Alcohol and other drugs


Chief executive officers


Continuous quality improvement


Non-government organisation


New South Wales


Primary Health Network


  1. United Nations. 3.5 Strengthen the prevention and treatment of substance abuse, including narcotic drug abuse and harmful use of alcohol. 2020 [cited 2020 15 June]; Available from:

  2. Gray D, Cartwright K, Stearne A, Saggers S, Wilkes E, Wilson M. Review of the harmful use of alcohol among Aboriginal and Torres Strait Islander people. Aust Ind HealthInfoNet. 2018;18(1):42. Retrieved on 12/12/2020 from:

  3. Australian Institute of Health and Welfare. Alcohol treatment guidelines. 2019 [cited 2020 15 June]; Available from:

  4. Kirmayer LJ. Rethinking cultural competence. Transcult Psychiatr. 2012;49(2):149–64.

    Article  Google Scholar 

  5. Downing R, Kowal E, Paradies Y. Indigenous cultural training for health workers in Australia. Int J Qual Health Care. 2011;23(3):247–57.

    Article  PubMed  Google Scholar 

  6. Williams DR, Lawrence JA, Davis BA. Racism and health: evidence and needed research. Annu Rev Public Health. 2019;40(1):105–25.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Wallace, R, and J Allan. NADA Practice Resource: Alcohol & other Drugs Treatment Guidelines for Working with Aboriginal & Torres Strait Islander People In a Non-Aboriginal Setting. 2019 [cited 2020 29 June]; Available from:

  8. Callejas, L.M. and M. Hernandez, Reframing the Concept of Cultural Competence to Enhance Delivery of Behavioral Health Services to Culturally Diverse Populations, in Foundations of Behavioral Health, B.L. Levin and A. Hanson, Editors. 2020, Springer International Publishing: Cham p 321-335.

  9. Guerrero EG, Khachikian T, Frimpong JA, Kong Y, Howard DL, Hunter S. Drivers of continued implementation of cultural competence in substance use disorder treatment. J Subst Abus Treat. 2019;105:5–11.

    Article  Google Scholar 

  10. Chae D, et al. Effectiveness of cultural competence educational interventions on health professionals and patient outcomes: A systematic review. Jpn J Nurs Sci. 2020:e12326.

  11. Saha S, Korthuis PT, Cohn JA, Sharp VL, Moore RD, Beach MC. Primary care provider cultural competence and racial disparities in HIV care and outcomes. J Gen Intern Med. 2013;28(5):622–9.

    Article  PubMed  PubMed Central  Google Scholar 

  12. Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM, et al. A refined compilation of implementation strategies: results from the expert recommendations for implementing change (ERIC) project. Implement Sci. 2015;10(1):21.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Dark F, Whiteford H, Ashkanasy NM, Harvey C, Crompton D, Newman E. Implementing cognitive therapies into routine psychosis care: organisational foundations. BMC Health Serv Res. 2015;15(1):310.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Centre for Epidemiology and Evidence. Study Design for Evaluating Population Health and Health Service Interventions: A Guide. In: Evidence and Evaluation Guidance Series, P.a.P.H. Division. Sydney: NSW Ministry of Health; 2019.

    Google Scholar 

  15. Eslinger J, Sprang G, Ascienzo S, Silman M. Fidelity and sustainability in evidence-based treatments for children: an investigation of implementation determinants. J Fam Soc Work. 2020;23(2):177–96.

    Article  Google Scholar 

  16. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4(1):50.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Wilson A. Addressing uncomfortable issues: reflexivity as a tool for culturally safe practice in Aboriginal and Torres Strait islander health. Aust J Indi Educ. 2014;43(2):218–30.

    Article  Google Scholar 

  18. Kirsh SR, Lawrence RH, Aron DC. Tailoring an intervention to the context and system redesign related to the intervention: a case study of implementing shared medical appointments for diabetes. Implement Sci. 2008;3(1):34.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Schierhout G, Brands J, Bailie R. Audit and Best Practice for Chronic Disease Extension Project, 2005–2009: Final Report PDF. Melbourne: The Lowitja Institute; 2010.

    Google Scholar 

  20. Munro A, Shakeshaft A, Clifford A. The development of a healing model of care for an indigenous drug and alcohol residential rehabilitation service: a community-based participatory research approach. Health Just. 2017;5(1):12.

    Article  Google Scholar 

  21. Knight A, Maple M, Shakeshaft A, Shakehsaft B, Pearce T. Improving the evidence base for services working with youth at-risk of involvement in the criminal justice system: developing a standardised program approach. Health Just. 2018;6(1):8.

    Article  Google Scholar 

  22. Wallerstein N, Duran B. Using community-based participatory research to address health disparities. Health Promot Pract. 2006;7(3):312–23.

    Article  PubMed  Google Scholar 

  23. Parry GJ, Carson-Stevens A, Luff DF, McPherson ME, Goldmann DA. Recommendations for evaluation of health care improvement initiatives. Acad Pediatr. 2013;13(6 Suppl):S23–30.

    Article  PubMed  Google Scholar 

  24. QSR International, NVivo qualitative data analysis software [software]. 1999.

  25. Microsoft Corporation, Microsoft excel. 2018.

  26. Bainbridge R, et al. Cultural competency in the delivery of health services for indigenous people. Canberra: Australian Government; 2015.

    Google Scholar 

  27. Chong A, et al. Improving cultural sensitivity to indigenous people in Australian hospitals : a continuous quality improvement approach. Focus Health Prof Educ. 2011;13(1):84–97.

    Google Scholar 

  28. Callejas L.M., Hernandez M. Reframing the Concept of Cultural Competence to Enhance Delivery of Behavioral Health Services to Culturally Diverse Populations. In: Levin B., Hanson A. (eds) Foundations of Behavioral Health. Cham: Springer; 2020.

  29. Fixsen D, et al. Implementation reseach: A synthesis of the literature. Tampa: University of South Florida, Louis de la parte Florida mental health Insitute, the National Implementation Research Network (FMHI publication #231); 2005.

    Google Scholar 

  30. Batalden PB, Davidoff F. What is "quality improvement" and how can it transform healthcare? Qual Saf Health Care. 2007;16(1):2–3.

    Article  PubMed  PubMed Central  Google Scholar 

  31. Bailie R, et al. Audit and best practice for chronic disease—project final report. Darwin: Menzies School of Health Research; 2007.

    Google Scholar 

  32. Bailie R, et al. Indigenous health: effective and sustainable health services through continuous quality improvement. Med J Aust. 2007;186(10):525–7.

    Article  PubMed  Google Scholar 

Download references


We would like to acknowledge and pay respects to the traditional custodians of the various lands where this project was completed. We would also like to acknowledge the valuable input of the Aboriginal Project Advisory Group, the Aboriginal Drug and Alcohol Network of NSW (ADAN), NSW Aboriginal Residential Healing Drug and Alcohol Network (NARHDAN), the Network of Alcohol and other Drugs Agencies (NADA) and the NGO AoD services and staff who took part.


The Alcohol and Other Drugs Treatment Guidelines for Working with Aboriginal and Torres Strait Islander people in a non-Aboriginal setting project was initiated and funded by a coalition of six Primary Health Networks (PHNs) in NSW, Australia. The project arose from the intention by the six PHNs to enhance and standardise culturally responsive practices at NGO AoD Treatment Services. The PHNs identified and contacted potential services within their jurisdictions regarding participation and provided contact details of the services to the project team. Funding was provided to NDARC to conduct the project evaluation. The PHNs had no role in the project design, implementation, data collection, analysis, interpretation of the data, or in the writing of this manuscript.

Author information

Authors and Affiliations



The project was overseen by the Network of Alcohol and other Drugs Agencies (NADA), implemented by the project team (RW and JA), and the evaluation was completed by the National Drug and Alcohol Research Centre (SF, AS and AA). NADA, the project team, the Aboriginal Project Advisory Group and the evaluation team all worked closely to develop, implement and evaluate the five-step process (RW, JA, SF, AS, AA), reporting to the coalition of Primary Health Networks (PHNs) who funded the project. RW, JA and SF acquired the data; SF conducted the data analysis; SF and AA wrote the methods and results, JA, AA, SF and AS wrote the introduction and discussion, with input from all authors. All authors provided substantial contributions to the interpretation of the results, and all critically revised the manuscript, and approved the final manuscript as submitted. The authors read and approved the final manuscript.

Corresponding author

Correspondence to Julaine Allan.

Ethics declarations

Ethics approval and consent to participate

Ethical approval was provided by the Aboriginal Health and Medical Research Council of NSW [#1487/19] and UNSW Human Research Ethics Committee [REC/16/CIPHS/46]. CEOs/senior management of all participating services and all staff participating in interviews and/or surveys provided informed consent to participate. All methods were performed in accordance with the relevant guidelines and regulations.

Consent for publication

Not applicable.

Competing interests

AA – no competing interests.

AS – no competing interests.

JA – no competing interests.

RW – worked for one of the participating services.

SF - no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: Appendix A.

Implementation workshop feedback survey items. Appendix B. Evaluation Framework and Program Logic - Project level. Appendix C. Example excerpt from a baseline service audit report.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Farnbach, S., Allan, J., Wallace, R. et al. Formative process evaluation of a guideline-driven process for improving the cultural responsiveness of alcohol and drug treatment services. BMC Health Serv Res 21, 352 (2021).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: