Skip to main content

Development and usability testing of tools to facilitate incorporating intersectionality in knowledge translation

Abstract

Background

The field of knowledge translation (KT) has been criticized for neglecting contextual and social considerations that influence health equity. Intersectionality, a concept introduced by Black feminist scholars, emphasizes how human experience is shaped by combinations of social factors (e.g., ethnicity, gender) embedded in systemic power structures. Its use has the potential to advance equity considerations in KT. Our objective was to develop and conduct usability testing of tools to support integrating intersectionality in KT through three key phases of KT: identifying the gap; assessing barriers to knowledge use; and selecting, tailoring, and implementing interventions.

Methods

We used an integrated KT approach and assembled an interdisciplinary development committee who drafted tools. We used a mixed methods approach for usability testing with KT intervention developers that included semi-structured interviews and the System Usability Scale (SUS). We calculated an average SUS score for each tool. We coded interview data using the framework method focusing on actionable feedback. The development committee used the feedback to revise tools, which were formatted by a graphic designer.

Results

Nine people working in Canada joined the development committee. They drafted an intersectionality primer and one tool that included recommendations, activities, reflection prompts, and resources for each of the three implementation phases. Thirty-one KT intervention developers from three countries participated in usability testing. They suggested the tools to be shorter, contain more visualizations, and use less jargon. Average SUS scores of the draft tools ranged between 60 and 78/100. The development committee revised and shortened all tools, and added two, one-page summary documents. The final toolkit included six documents.

Conclusions

We developed and evaluated tools to help embed intersectionality considerations in KT. These tools go beyond recommending the use of intersectionality to providing practical guidance on how to do this. Future work should develop guidance for enhancing social justice in intersectionality-enhanced KT.

Peer Review reports

Contributions to the literature

  • This paper describes the development of the first tools for explicitly integrating intersectionality in KT planning. It brings together symbiotic work from two important and historically distinct academic traditions with shared goals of advancing social conditions (which includes health) and justice.

  • The findings of the project move beyond recommending the use of intersectionality to providing practical resources on how to do this. The resultant tools may help ensure that KT interventions are inclusive and have the most potential to help a greater proportion of people.

Background

Complexity and context are recognized as critical for implementation of evidence-informed health services and policies [1, 2]. Among many complex contextual factors, social factors, “the conditions in which people are born, grow, work, live, and age, and the wider set of forces and systems shaping the condition of daily life” [3], are consistently identified as key determinants of health and health inequities [4, 5]. Implementation science, and the field of knowledge translation (KT) more broadly, has been criticized for neglecting social factors such as gender [6] and equity [7] and perpetuating inequities due to foundational evidence that underrepresents key populations such as women and racialized communities [8]. Numerous efforts are underway that address these criticisms, including calls for explicit consideration of equity in every implementation project [9], synthesis of implementation equity effectiveness data, adaptation of conceptual frameworks, and development of guidance to support integrating equity considerations specifically in implementation research and practice [10,11,12] and more broadly in evidence-based decision-making to address global societal challenges [13].

Intersectionality is one approach that can advance equity considerations in KT and implementation. Intersectionality, a term coined by Dr. Kimberle Crenshaw [14], is a theoretical approach based on the work of Black feminist and critical race scholars who identified and emphasized how Black women faced multiple and intersecting forms of oppression [15,16,17]. In the context of health, Hankivsky and colleagues described how intersectionality “focuses on the ways in which multiple axes of social inequality intersect and co-construct one another at the macro and micro levels to produce a broad range of unequal outcomes, in both individual and population health” [18]. Axes of social inequality include (but are not limited to) experiences of racism, classism, heteronormativity, cisnormativity, ageism, audism, and xenophobia. The social intersections inhabited by an individual are not inherently additive [19] and reflect larger systems of privilege and/ or exclusion [20]. As such, intersectionality explains how a person’s experiences of oppression and/ or privilege may shift depending on the specific situation and highlights how these experiences and inequities unfold within complex health and social system structures (Fig. 1).

Fig. 1
figure 1

Visual representation of some intersecting categories. The categories mentioned in this figure are not an exhaustive list

We propose that taking an intersectional approach may improve the design, delivery, receipt, effectiveness, and impact of KT and implementation interventions by more directly addressing specific health inequities affecting individuals and groups. Conversely, neglecting intersecting social factors in KT and implementation may perpetuate inequities leading to poor health outcomes [21]. For example, an analysis of census data in Ontario, Canada showed that neighbourhoods with a higher concentration of people of colour during the COVID-19 pandemic had three times higher COVID-19 infection rates, four times higher hospitalization rates, and deaths twice as high compared to neighbourhoods with lower concentrations of people of colour [22]. People of colour are more likely to be employed in work most negatively affected by the pandemic (e.g., food services); newcomers are also overrepresented in jobs with greater potential exposure to COVID-19 [22]. Accordingly, newcomers who are people of colour face compounded challenges in health and well-being due to intersections in social determinants of health [22]. If public health strategies do not use an intersectional lens, we risk that these populations will continue to bear the brunt of morbidity and mortality in future COVID-19 waves. However, despite arguments supporting the principles of intersectionality, intersectionality is critiqued for being difficult to apply [23]. Practical guidance and strategies for applying intersectionality in KT and implementation are warranted. As such, our objective in this project was to develop, evaluate and refine a suite of tools to support the application of intersectionality in KT. Our ultimate goal of developing these tools is to increase knowledge, facilitate positive attitudes, and support adoption of intersectional considerations in KT.

Methods

Project foundations and preliminary work

This study took place within a series of projects to advance intersectionality considerations in KT [24,25,26,27,28]. The series of projects took place in Canada between 2017 and 2019. A full description of the series of projects is available [27]. The projects used an integrated knowledge translation approach [29], defined here as a collaborative research process in which knowledge users [those who can use the knowledge generated through research to make informed decisions] [30] are engaged in governance and conduct of the research process. In the development phase, the overall project team led by SES identified KT intervention developers as the primary knowledge users for this work, and 19 people with experience developing KT interventions in research or practice roles were engaged as co-investigators or collaborators from project conception. Recognizing the interdisciplinary nature of this work, the project steering committee invited 26 scholars with training in intersectionality, women and gender studies, and health equity as co-investigators or collaborators. These individuals were identified from those who received national research funding or who published in peer-reviewed journals in this area.

The overall project team selected the Knowledge-to-Action (KTA) Framework [31], a highly cited implementation process model [32, 33] developed from a review of over 30 planned action theories, as an overarching framework. To facilitate the process of applying intersectionality in KT, the team chose to narrow our target context to health for older adults to reflect the needs of a population that has growing health care needs and where documented health disparities exist [34,35,36,37]. In the first project step, they invited a diverse group of community members, clinicians, KT practitioners, and researchers to participate in a Nominal Group Technique exercise and prioritized phases of KT to address using intersectionality [28]. In the second step, they prioritized the following KTA phases for integrating intersectionality: identifying the problem/ evidence-to-practice gaps; assessing barriers to knowledge use; and selecting, tailoring, and implementing interventions [28]. They then used a structured consensus process to identify one model, theory, or framework (MTF) corresponding to each prioritized KTA phase [25]. The following MTFs were selected: Iowa Model of Evidence-Based Practice [38] (identifying the problem/ evidence-to-practice gaps), the Consolidated Framework for Implementation Research [39] (assessing barriers to knowledge use), and the Theoretical Domains Framework/ Behaviour Change Wheel [40] (selecting, tailoring, and implementing interventions). In the third step, they developed intersectionality-enhanced versions of each selected MTF through an iterative process of co-creation and consensus-building [24, 27].

Study design

We used an integrated knowledge translation approach with an iterative, multi-step process to develop the suite of tools for applying intersectionality in each of the prioritized KTA phases and accompanying intersectionality-enhanced MTFs (Fig. 2). Our integrated knowledge translation approach involved including the project’s primary knowledge users (KT intervention developers) as members of an interdisciplinary committee we established to develop the tools (described below). All committee members were provided the opportunity to meet international standards for authorship of this peer-reviewed manuscript [41].

Fig. 2
figure 2

Tool Development Process

Development committee formation and approach

We established an interdisciplinary committee to lead the tool development process. We recruited development committee members through informal networks of existing overall project team members (academic and knowledge user co-investigators and collaborators) and purposive recruitment of intersectionality experts. All committee members, including knowledge users, were involved at a level consistent with Collaborate on the IAP2 Spectrum of Public Participation, whereby members partnered “in each aspect of the decision, including the development of alternatives and the identification of the preferred solution” [42].

We created Terms of Reference for the development committee (Appendix A, see in Additional file 1). The committee met on an ad hoc basis via web-based conferencing system. All committee members provided their time in-kind. As intersectionality emphasizes reflexivity as a core concept, the committee worked to model intersectionality practice during each meeting, guided by Weaver’s Long Table Format [43]. For example, in each meeting, each member introduced themselves and specified how they would like to be addressed, what pronouns to use, and why they were interested in the project. Committee members were also encouraged to independently reflect on their personal values, social locations, privileges and impact of all these factors on their work in this project, and to complete the Harvard Implicit Association test [44].

Step 1: draft tool development

The development committee members held a series of meetings to develop draft tools. During initial meetings, committee members reviewed background project information and reference materials for each prioritized KTA phase and accompanying intersectionality-enhanced MTF and discussed ideas for scope, content, and structure of the suite of tools. The committee agreed that decisions would be made by consensus when at least 50% of committee members were present; if a decision by consensus was not possible, meeting attendees voted. Committee members identified a short list of tool options, discussed format and content options, and decided on the tool components. Research staff then created draft tools based on the instructions and guidance of the committee. Committee members individually reviewed and suggested revisions for each tool and discussed these revisions at subsequent meetings. This process was repeated until all members were satisfied with the tool drafts and did not request additional revisions. Following each committee discussion, research staff revised draft tools based on committee guidance and input from graphic designers.

Step 2: usability testing

We invited individuals who represented the relevant knowledge user group and self-identified as having experience developing KT interventions to participate using a purposive sampling approach via individual emails, recruitment flyers, electronic newsletters, and Twitter. We included individuals professionally employed in KT practice, as well as KT and implementation scientists. We recruited eight to ten individuals to test each tool, consistent with previous qualitative research sampling approaches [45].

Participants reviewed the tool prior to the interview and had access to the tool during the interview (either on screen or paper copy). Interview participants were sent questions in advance. Open-ended questions explored participant perceptions of each tool during one-hour semi-structured interviews. The interview included probes on issues that emerged, including the content and structure of the tools (Appendix B, see in Additional file 2). Feedback was elicited using the “think aloud” method [46]. Specifically, participants described thoughts, questions or concerns that came to mind as they reviewed the tool during the interview and reflected on how they would use it in practice. Interviews were conducted in English by two members of the research team (an interviewer and a note taker). A detailed notetaking chart to track information related to participants’ overall comments and suggestions for change was used during the interview. Each participant was asked to complete the System Usability Scale (SUS), a reliable usability measure [47, 48]. The SUS consists of 10 questions on a 5-point Likert scale, which is interpreted as an overall score between 0 and 100 [47, 48]. A score exceeding 68 is considered above average usability [47]. Interviews were audio recorded.

Immediately after each interview, the interviewer and note taker discussed notes and collated comments to ensure all relevant insights were recorded. If any points were unclear, the team reviewed the audio recording. They reviewed and resolved any conflicts in the notes until no critical issues were identified. They organized data using a framework approach [49]. For each tool, they coded interview findings into four categories based on potential actionability of the feedback: i. no action required (e.g., “this tool will be especially useful for less experienced team members); ii. formatting (e.g., “it would be helpful if key questions were bolded); iii. out of scope & no action taken (e.g., “it would be great if you can make a video explaining different sections of the tool”); and, iv. discussion and decision required (e.g., “is it appropriate to use the term Indigenous ancestry?”). Any feedback related to the substantive content of the tool was coded for discussion and decision. The two staff who conducted interviews coded data independently and resolved conflicts through discussion [49].

Step 3: tool refinement

For each tool, research staff created a summary report of the interview feedback and presented it to the development committee. To facilitate review and discussion, staff identified related or commonly mentioned issues and organized them into groups. Committee members individually reviewed summary reports, then met to address the discussion feedback; grouped issues were summarized in slides and discussed by the committee. As with the previous phase, decisions were made when at least 50% of meeting attendees voted in favor of a decision. When the committee did not meet the threshold for a decision, comments were forwarded to the principal investigator (SES, also a knowledge user) for final decision. Research support staff then revised tool content based on committee guidance and a copy editor and graphic designer finalized each tool’s wording, formatting, and layout.

Results

Development committee structure and processes

Eleven individuals joined the development committee; however, two individuals withdrew before the first meeting due to other commitments. Committee members included a subset of project co-investigators (n = 3) and members identified through external recruitment (n = 6). The development committee consisted of scholars with training in women and gender studies, political science and intersectionality, adult education experts, KT research funders, KT practitioners and KT and implementation researchers. Most individuals on the 9-member committee identified as white, heterosexual, female, had no spiritual or religious affiliation, were married or cohabitating, spoke English as a first language, had a masters or doctoral degree, lived in a medium or large population centre, in their own home, had full-time employment, and an annual income > 90,000$ CAD. The committee members were distributed across 4 out of 13 Canadian provinces. Six individuals represented the relevant knowledge user group of KT intervention developers (five as practitioners, one as researcher). Three committee members chose to be involved in manuscript writing and are included as co-authors.

Draft tool development

The committee held twelve meetings over 1 year during the tool development phase. Initial discussions addressed communication practices and reflections on self-disclosed diversity within the development committee. The committee acknowledged that limited intersecting social categories were represented on the committee. Committee members also emphasized the need for practical tools and to “not reinvent the wheel” and suggested highlighting existing intersectionality resources and references. The committee supported the use of case studies or examples within each tool in addition to reflection questions and intersectionality considerations. Committee members recommended the need for a tool that introduced intersectionality, recognizing that many KT intervention developers may be unfamiliar with the foundations of intersectionality. An early decision of the committee was to focus on the three KTA phases, and include the intersectionality-enhanced MTFs as companions for each phase. The committee made this decision in recognition of the overall guiding framework for the project and the need for practical resources.

Based on these initial discussions and decisions (detailed in Appendix C, see in Additional file 3), the committee developed drafts for four distinct and related tools: an introductory primer on intersectionality (“Intersectionality Guide”) and one tool to support each of the three prioritized KTA phases. For each tool, two to three major revisions were undertaken to produce a draft tool that went forward to usability testing. For the phase Identifying the problem/ evidence-to-practice gap, the committee created a “Reflection Workbook” tool to prompt users to reflect on key principles and issues of intersectionality and to consider how that influences KT practice decisions. For the phase Assessing barriers to knowledge use, the committee created a “Guide to Common Approaches for Assessing Barriers and Facilitators” that focused on prompts and approaches for including intersectional considerations within data collection methods. For the phase Selecting, tailoring and implementing interventions, the committee created a workbook to support a structured approach to developing and conducting KT interventions with an intersectional lens. Draft tools ranged in length from 10 to 25 pages and contained case studies, intersectionality reflection questions, and resources on intersectionality and implementation.

Usability testing

Thirty-one individuals with experience developing KT interventions participated in usability testing (n = 7–9 people/ tool). Characteristics for 27 participants are described in Table 1. The intervention developers had an average of 9 years of KT experience working across three different countries.

Table 1 Usability testing participant characteristics

Across the four draft tools, 22 issues were coded as needing discussion and a decision by the development committee (average of 6/ tool). All issues are reported in Appendix D (column A) (see in Additional file 4), and here we summarize issues noted across all tools and highlight issues that stimulated significant discussion and revision by the committee.

Consistent feedback received across all tools was the need to shorten length and include more visualizations. There were also multiple requests for additional information, resources and suggestions for further elaboration. There were numerous comments to clarify and simplify the language level and terms used in the tools, which several participants felt contained unnecessary jargon. Participants requested clear reporting about decisions made (e.g., rationale for use of specific definitions) and underlying assumptions for the tools (e.g., expectation that user has some existing knowledge of KT practice).

There were several issues that subsequently sparked significant committee discussion. Several participants commented on the use of specific terms (e.g., privilege, disadvantage, oppression) and debated whether they might serve to engage the intended knowledge user audience, or alternatively, result in a negative reaction that may discourage some participants from using the tools. Participants raised the specific issue of privilege and emphasized the need to ensure the tools do not further exclude individuals who experience oppression. Participants also commented on the intersectional considerations emphasized in the tools. For example, the importance of considering Indigenous ancestry in the context of intersectionality was raised.

Specific feedback was received on the tool for the Selecting, tailoring, and implementing interventions phase. In particular, participants felt that the tool was overwhelming due to the significant amount of content. Participants felt that intersectionality considerations were most relevant in the selecting and tailoring implementation interventions sub-components of the phase and recommended removing the section focused on implementing interventions because it did not contain information specific to intersectionality.

Average SUS scores are summarized in Table 2. Only one tool draft was considered above average in terms of usability (Guide for Common Approaches to Assessing Barriers & Facilitators to Knowledge Use). The remaining tool drafts received a below average usability rating.

Table 2 Average System Usability Scale (SUS) summary scores for each draft tool component

Tool refinement

The development committee met four times to address the feedback. The committee reduced the length of individual tools and discussed specific strategies for shortening each tool (e.g. moving some content to appendices, refining table of contents). The committee created short (one-two pages) summaries of longer tools (i.e., those more than ten pages). A statement declaring project assumptions and incoming biases was added to the introduction of each tool. A statement was also created on Indigeneity and the limitations of project considerations and incorporated into each tool. Tool components considered to potentially oppress some individuals further (e.g., social identities statements activity) were included as optional resources/ references only or removed.

The committee discussed the recommendation to remove the “implement intervention” component from the “select, tailor, implement” tool at length, recognizing that this represented a distinct shift from the KTA Framework on which the project was based. The committee decided to move the “implement” content to an appendix to simplify and facilitate deeper consideration of intersectionality in selecting and tailoring KT interventions.

The final suite of tools included six documents: i) Intersectionality Guide; ii) Intersectionality and Knowledge Translation Guide Sheet; iii) Reflection Workbook; iv) Reflection Worksheet; v) Guide for Common Approaches to Assessing Barriers & Facilitators to Knowledge Use; vi) Selecting and Tailoring KT Interventions Workbook [27]. Details of the tools are described in Supplementary Table 1 and all tools are available in Appendix E.

Discussion

We used a collaborative, reflective, comprehensive and rigorous approach to create a suite of tools to support the development of KT interventions that consider intersectionality. Our collaborative and reflective approach required us to be both responsive and iterative. For example, the development committee chose to focus the tools on each prioritized KTA phase rather than the associated intersectionality-enhanced MTFs. We see this focus as a strength that increases tools’ applicability because users are not limited to using the resources with a specific MTF. A second key point of response was the development committee’s decision to revise the focus of one tool on selecting and tailoring implementation interventions, reducing the emphasis on implementation of interventions, in response to usability test feedback.

Our comprehensive and rigorous approach is consistent with other tool development methods reported in implementation science [50, 51]. A framework for implementation tool development published after this project was launched emphasizes several components included in our development process, such as explore (define), develop, and integrated knowledge translation. We are also addressing evaluation and action planning through our ongoing work evaluating the effects of the tools when applied in an established KT training program. Our tools are distinguished from existing intersectionality guides [52, 53] in two important ways: first, the explicit focus on incorporating intersectionality in KT and implementation research and practice; and second, the explicit reporting of our methodology and approach for tool development. While other intersectionality guides have been identified as using a collaborative approach [52], few (if any) details are reported. There may be different standards for development and reporting for guides across disciplines, and while recognized as important in implementation science, it should not be viewed as a limitation of other guides. An important strength of this project is our interdisciplinary approach, in which we are among the first to bring together people working in both intersectionality and knowledge translation. Team and committee members addressed different foundational knowledge bases, biases and interpretations and were able to come together to produce the tools. This process was not straightforward, and we discuss in a companion paper the challenges, opportunities and tensions we worked through as a team [26].

We recognize the limitations of tools and the series of projects as a whole. Our overall project does not address all components of intersectionality equally. Prior to the current phase of tool development, the overall project team decided to focus on increasing consideration of intersecting social factors in KT, prioritizing this as an appropriate “first step” for our target knowledge user group (KT intervention developers). Other important features of intersectionality, in particular, power, privilege and social justice, were addressed through an embedded approach within the tools. We acknowledge known criticisms of our type of approach. Rice and colleagues argued that “it does intellectual disservice to coopt the parts of a theory that are compatible with existing structures and power relations (for the technical work the theory does in managing complexity) while ignoring aspects of the theory that orient to intervening in those structures and relations to liberatory ends” [23]. The overall project team’s decision to focus on intersecting social considerations may relate in part to the historical traditions of implementation science that do not explicitly consider social justice [54], and may reflect implicit biases towards aspects of intersectionality that are more congruent with implementation conventions. We maintain that even this introductory approach to intersectionality has disruptive potential for KT and implementation science. We recognize that while it is not always necessary to always address all intersections, it can be challenging to determine which components are relevant to the topic at hand, and thus tools need to support the full complement of intersectionality. We will continue to address this limitation in future work, and in the interim refer readers to complementary intersectionality guides with a stronger focus on advocacy and social justice [26].

A second important limitation of our work was that despite our efforts to engage diverse perspectives, not all voices are represented. We recognize that those involved in the project represent a limited range of privileged identities. We acknowledge that our tools were developed with mostly Canadian input, and entirely from high income countries. We recognize that we did not address Indigeneity, which is important in light of Canada’s colonial history (and relevant around the world). We address this limitation in detail on our project website [55]. The limited extent of Indigenous considerations has also been recognized in other intersectionality tools [56], which may reflect ongoing debates about the challenges associated with talking about Indigeneity using intersectionality, as there are questions as to whether this framework is appropriate for addressing colonial legacies and Indigenous values [57, 58]. Lastly, we recognize our methodological limitations whereby we only conducted a single round of usability testing due to financial constraints, and that we did not transcribe interviews due to budget limitations. However, interviews were recorded and reviewed by two staff members to identify salient data.

Conclusions

Our work makes important advancements to KT. The project as a whole has advanced explicit theoretical considerations of intersectionality in established frameworks, and in KT intervention planning. We developed practical tools that can be used by KT intervention developers to embed intersectionality into KT initiatives, going beyond recommending the use of intersectionality to providing practical guidance on how to do this. Our tools may help ensure that KT planning and resultant interventions are inclusive and have the most potential to help the most people. We invite readers to use the tools and report on their experiences with them so that we can learn, improve future iterations, and examine their effectiveness.

Availability of data and materials

The datasets used and analyzed during the current study are available from the corresponding author on reasonable request.

Abbreviations

KTA:

Knowledge-to-Action

KT:

Knowledge Translation

MTF:

Model, Theory, or Framework

SUS:

System Usability Scale

References

  1. Braithwaite J, Churruca K, Long JC, Ellis LA, Herkes J. When complexity science meets implementation science: a theoretical and empirical analysis of systems change. BMC Med. 2018;16(1):63. https://doi.org/10.1186/s12916-018-1057-z Available from: https://bmcmedicine.biomedcentral.com/articles/10.1186/s12916-018-1057-z#citeas. [cited 2021 Feb 22].

    Article  PubMed  PubMed Central  Google Scholar 

  2. Pfadenhauer LM, Gerhardus A, Mozygemba K, Lysdahl KB, Booth A, Hofmann B, et al. Making sense of complexity in context and implementation: the context and implementation of complex interventions (CICI) framework. Implementation Sci. 2017;12(1):21. https://doi.org/10.1186/s13012-017-0552-5 Available from: https://implementationscience.biomedcentral.com/articles/10.1186/s13012-017-0552-5#citeas. [cited 2021 Feb 22].

    Article  Google Scholar 

  3. World Health Organization. Social determinants of health. [place unknown]: World Health Organization; c2021. Available from: https://www.who.int/teams/social-determinants-of-health. [cited 2021 Feb 22].

  4. Braveman P, Gottlieb L. The social determinants of health: it’s time to consider the causes of the causes. Public Health Rep. 2014;129 Suppl 2(Suppl 2):19–31. https://doi.org/10.1177/00333549141291S206 Available from: https://journals.sagepub.com/doi/pdf/10.1177/00333549141291S206. [cited 2021 Feb 22].

    Article  PubMed  Google Scholar 

  5. Sorensen G. Introduction to social determinants of health. In: Goldman MB, Harch MC, editors. Women and health. San Diego: Academic Press; 2000. p. 523–7. Available from: https://www.sciencedirect.com/science/article/pii/B9780122881459500486. [cited 2021 Feb 22].

    Chapter  Google Scholar 

  6. Tannenbaum C, Greaves L, Graham ID. Why sex and gender matter in implementation research. BMC Med Res Methodol. 2016;16(1):145. https://doi.org/10.1186/s12874-016-0247-7 Available from: https://bmcmedresmethodol.biomedcentral.com/articles/10.1186/s12874-016-0247-7#citeas. [cited 2021 Feb 22].

    Article  PubMed  PubMed Central  Google Scholar 

  7. Casey AE. Foundation, National implementation research network. Bringing equity to implementation: incorporating community experience to improve outcomes. Stanf Soc Innov Rev. 2021; Available from: http://stanford.ebookhost.net/ssir/digital/75/ebook/1/download.pdf. [cited 2021 Jul 5].

  8. Baumann AA, Cabassa LJ. Reframing implementation science to address inequities in healthcare delivery. BMC Health Serv Res. 2020;20(1):190. https://doi.org/10.1186/s12913-020-4975-3 Available from: https://bmchealthservres.biomedcentral.com/articles/10.1186/s12913-020-4975-3. [cited 2021 Jul 5].

    Article  PubMed  PubMed Central  Google Scholar 

  9. Brownson RC, Kumanyika SK, Kreuter MW, Haire-Joshu D. Implementation science should give higher priority to health equity. Implement Sci. 2021;16(1):28. https://doi.org/10.1186/s13012-021-01097-0 Available from https://implementationscience.biomedcentral.com/articles/10.1186/s13012-021-01097-0. [cited 2021 Jul 5].

    Article  PubMed  PubMed Central  Google Scholar 

  10. Woodward EN, Matthieu MM, Uchendu US, Rogal S, Kirchner JE. The health equity implementation framework: proposal and preliminary study of hepatitis C virus treatment. Implement Sci. 2019;14(1):26. https://doi.org/10.1186/s13012-019-0861-y Available from: https://implementationscience.biomedcentral.com/articles/10.1186/s13012-019-0861-y. [cited 2022 Mar 11].

    Article  PubMed  PubMed Central  Google Scholar 

  11. Harding T, Oetzel J. Implementation effectiveness of health interventions for indigenous communities: a systematic review. Implement Sci. 2019;14(1):76. https://doi.org/10.1186/s13012-019-0920-4 Available from: https://implementationscience.biomedcentral.com/articles/10.1186/s13012-019-0920-4. [cited 2022 Mar 11].

    Article  PubMed  PubMed Central  Google Scholar 

  12. Woodward EN, Singh RS, Ndebele-Ngwenya P, Melgar Castillo A, Dickson KS, Kirchner JE. A more practical guide to incorporating health equity domains in implementation determinant frameworks. Implement. Sci Commun. 2021;2(1):61. https://doi.org/10.1186/s43058-021-00146-5 Available from: https://implementationsciencecomms.biomedcentral.com/articles/10.1186/s43058-021-00146-5. [cited 2022 Mar 11].

    Article  Google Scholar 

  13. Global Commission on Evidence to Address Societal Challenges. The Evidence Commission report: A wake-up call and path forward for decision- makers, evidence intermediaries, and impact-oriented evidence producers: Hamilton, McMaster Health Forum; 2022. Available from: https://www.mcmasterforum.org/docs/default-source/evidence-commission/evidence-commission-report.pdf? Status=Master&sfvrsn=2fb92517_5/Evidence-Commission-report

    Google Scholar 

  14. Hooks B. Ain’t I a woman: black women and feminism. Cambridge: South End Press; 1981.

    Google Scholar 

  15. Lorde A. Sister outsider: essays and speeches by Audre Lorde. Toronto: Crossing Press; 2007.

    Google Scholar 

  16. Crenshaw K. Mapping the margins: Intersectionality, identity politics, and violence against women of color. Stanford Law Rev. 1991;43(6):1241–99. https://doi.org/10.2307/1229039 Available from: https://www.jstor.org/stable/1229039?seq=1. [cited 2021 Feb 22].

    Article  Google Scholar 

  17. Crenshaw K. Demarginalizing the intersection of race and sex: a black feminist critique of antidiscrimination doctrine, feminist theory and antiracist politics. Univ Chic Leg Forum. 1989;1989(1):139–67 Available from: https://chicagounbound.uchicago.edu/cgi/viewcontent.cgi?article=1052&context=uclf. [cited 2021 Jul 5].

    Google Scholar 

  18. Hankivsky O, Grace D, Hunting G, Giesbrecht M, Fridkin A, Rudrum S, et al. An intersectionality-based policy analysis framework: critical reflections on a methodology for advancing equity. Int J Equity Health. 2014;13:119. https://doi.org/10.1186/s12939-014-0119-x Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4271465/. [cited 2021 Feb 22].

    Article  PubMed  PubMed Central  Google Scholar 

  19. Bauer GR. Incorporating intersectionality theory into population health research methodology: challenges and the potential to advance health equity. Soc Sci Med. 2014;110:10–7. https://doi.org/10.1016/j.socscimed.2014.03.022 Available from: https://www.sciencedirect.com/science/article/pii/S0277953614001919?via%3Dihub. [cited 2021 Feb 22].

    Article  PubMed  Google Scholar 

  20. Nixon SA. The coin model of privilege and critical allyship: implications for health. BMC Public Health. 2019;19(1):1637. https://doi.org/10.1186/s12889-019-7884-9 Available from: https://bmcpublichealth.biomedcentral.com/articles/10.1186/s12889-019-7884-9. [cited 2021 Feb 22].

    Article  PubMed  PubMed Central  Google Scholar 

  21. Warner DF, Brown TH. Understanding how race/ethnicity and gender define age-trajectories of disability: An intersectionality approach. Soc Sci Med. 2011;72(8):1236–48. https://doi.org/10.1016/j.socscimed.2011.02.034 Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3087305/. [cited 2021 Feb 22].

    Article  PubMed  PubMed Central  Google Scholar 

  22. Statistics Canada. Impacts on immigrants and people designated as visible minorities. Ottawa: Statistics Canada; 2020. Available from: https://www150.statcan.gc.ca/n1/pub/11-631-x/2020004/s6-eng.htm. [updated 2021 Mar 4; cited 2021 Jul 7]

    Google Scholar 

  23. Rice C, Harrison E, Friedman M. Doing justice to intersectionality in research. Cult Stud Crit Methodol. 2019;19(6):409–20. https://doi.org/10.1177/1532708619829779 Available from: https://journals.sagepub.com/doi/abs/10.1177/1532708619829779?journalCode=csca#articleCitationDownloadContainer. [cited 2021 Feb 22].

    Article  Google Scholar 

  24. Etherington N, Rodrigues IB, Giangregorio L, Graham ID, Hoens AM, Kasperavicius D, et al. Applying an intersectionality lens to the theoretical domains framework: a tool for thinking about how intersecting social identities and structures of power influence behaviour. BMC Med Res Methodol. 2020;20(1):169. https://doi.org/10.1186/s12874-020-01056-1 Available from: https://bmcmedresmethodol.biomedcentral.com/articles/10.1186/s12874-020-01056-1. [cited 2021 Feb 22].

    Article  PubMed  PubMed Central  Google Scholar 

  25. Presseau J. Mobilizing an intersectional Lens in KT: enhancing the application of KT models, theories, and frameworks. Toronto, ON: Presented at: KT Canada Annual Scientific Meeting; 2021.

    Google Scholar 

  26. Kelly C, Kasperavicius D, Duncan D, Etherington N, Giangregorio L, Presseau P, et al. ‘Doing’ or ‘using’ intersectionality? Opportunities and challenges in incorporating intersectionality into knowledge translation theory and practice. Int J Equity Health. 2021;20(1):187. https://doi.org/10.1186/s12939-021-01509-z Available from: https://equityhealthj.biomedcentral.com/articles/10.1186/s12939-021-01509-z. [cited 2022 Mar 11].

    Article  PubMed  PubMed Central  Google Scholar 

  27. Knowledge Translation Program. Intersectionality & KT. Toronto: Knowledge Translation Program; c2019. Available from: https://knowledgetranslation.net/portfolios/intersectionality-and-kt/. [cited 2021 Feb 22]

  28. Kasperavicius D, Moore J, Bruyn-Martin L, Giangregorio L, Giguère A, Graham I, et al. Embedding intersectionality: building intersectional infrastructure for evidence-based implementation practitioners. Washington, DC: Poster presented at: 11th Annual Conference on the Science of Dissemination and Implementation in Health; 2018.

    Google Scholar 

  29. Kothari A, McCutcheon C, Graham ID. Defining integrated knowledge translation and moving forward: a response to recent commentaries. Int J Health Policy Manag. 2017;6(5):299–300. https://doi.org/10.15171/ijhpm.2017.15 Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5417154/. [cited 2021 Jul 13].

    Article  PubMed  PubMed Central  Google Scholar 

  30. Canadian Institutes of Health Research. Glossary of funding-related terms. Ottawa: Canadian Institutes of Health Research. Available from: https://cihr-irsc.gc.ca/e/34190.html. [updated 2021 Feb 16; cited 2021 Jul 5]

  31. Graham ID, Logan J, Harrison MB, Straus SE, Tetroe J, Caswell W, et al. Lost in knowledge translation: time for a map? J Contin Educ Heal Prof. 2006;26(1):13–24. https://doi.org/10.1002/chp.47 Available from: https://journals.lww.com/jcehp/Abstract/2006/26010/Lost_in_knowledge_translation__Time_for_a_map_.3.aspx. [cited 2021 Feb 22].

    Article  Google Scholar 

  32. Field B, Booth A, Ilott I, Gerrish K. Using the knowledge to action framework in practice: a citation analysis and systematic review. Implement Sci. 2014;9:172 Available from: https://implementationscience.biomedcentral.com/articles/10.1186/s13012-014-0172-2. [cited 2022 May 27].

    Article  Google Scholar 

  33. Skolarus TA, Lehmann T, Tabak RG, Harris J, Lecy J, Sales AE. Assessing citation networks for dissemination and implementation research frameworks. Implement Sci. 2017;12(1):97.

    Article  Google Scholar 

  34. Shim E. Regional variability in COVID-19 case fatality rate in Canada, February–December 2020. Int J Environ Res Public Health. 2021;18(4):1839. https://doi.org/10.3390/ijerph18041839 Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7918493/. [cited 2021 Jul 5].

    Article  PubMed  PubMed Central  Google Scholar 

  35. Public Health Agency of Canada. From risk to resilience: an equity approach to COVID-19. Ottawa: Public Health Agency of Canada; 2020. p. 86. Report No.: 200173. Available from: https://www.canada.ca/content/dam/phac-aspc/documents/corporate/publications/chief-public-health-officer-reports-state-public-health-canada/from-risk-resilience-equity-approach-covid-19/cpho-covid-report-eng.pdf. [cited 2021 Jul 5]

    Google Scholar 

  36. Flint AJ, Bingham KS, Iaboni A. Effect of COVID-19 on the mental health care of older people in Canada. Int Psychogeriatr. 2020;32(10):1113–6. https://doi.org/10.1017/S1041610220000708 Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7235298/. [cited 2021 Jul 5].

    Article  PubMed  Google Scholar 

  37. Jackson SE, Hackett RA, Steptoe A. Associations between age discrimination and health and wellbeing: cross-sectional and prospective analysis of the English Longitudinal Study of Ageing. Lancet Public Health. 2019;4(4):e200–8. https://doi.org/10.1016/S2468-2667(19)30035-0 Available from: https://www.thelancet.com/journals/lanpub/article/PIIS2468-2667(19)30035-0/fulltext. [cited 2021 Jul 5].

    Article  PubMed  Google Scholar 

  38. Titler MG, Kleiber C, Steelman VJ, Rakel BA, Budreau G, Everett LQ, et al. The Iowa Model of Evidence-Based Practice to promote quality care. Crit Care Nurs Clin North Am. 2001;13(4):497–509 Available from: https://pubmed.ncbi.nlm.nih.gov/11778337/. [cited 2021 Feb 22].

    CAS  Article  Google Scholar 

  39. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4:50. https://doi.org/10.1186/1748-5908-4-50 Available from: https://implementationscience.biomedcentral.com/articles/10.1186/1748-5908-4-50. [cited 2021 Feb 22].

    Article  PubMed  PubMed Central  Google Scholar 

  40. Michie S, van Stralen MM, West R. The behaviour change wheel: a new method for characterising and designing behaviour change interventions. Implement Sci. 2011;6:42. https://doi.org/10.1186/1748-5908-6-42 Available from: https://implementationscience.biomedcentral.com/articles/10.1186/1748-5908-6-42. [cited 2021 Feb 22].

    Article  PubMed  PubMed Central  Google Scholar 

  41. International Committee of Medical Journal Editors. Defining the role of authors and contributors. [place unknown]: International Committee of Medical Journal Editors; 2021. Available from: http://www.icmje.org/recommendations/browse/roles-and-responsibilities/defining-the-role-of-authors-and-contributors.html. [cited 2021 Jul 5].

  42. International Association for Public Participation. IAP2 spectrum of public participation. Denver: International Association for Public Participation; 2018. Available from https://cdn.ymaws.com/www.iap2.org/resource/resmgr/pillars/Spectrum_8.5x11_Print.pdf. [cited 2021 Jul 5]

    Google Scholar 

  43. The Long Table Etiquette Guide. [place known]: The Long Term Etiquette Guide. Available from: https://cdn.ymaws.com/ipayweb.org/resource/collection/54ECDF2C-12E6-43D2-8491-9568D2D2C958/LONG_TABLE_ETIQUETTE.pdf. [cited 2021 Feb 22].

  44. Project Implicit, Inc. Take a demo test. Boston: Project Implicit, Inc. Available from: https://implicit.harvard.edu/implicit/canada/selectatest.jsp. [cited 2021 Feb 22].

  45. Kuzel AJ. Sampling in qualitative inquiry. In: Crabtree BF, Miller WL, editors. Doing qualitative research. 2nd ed. Thousand Oaks (CA): Sage Publications; 1999. p. 33–45.

    Google Scholar 

  46. Kushniruk AW, Patel VL, Cimino JJ. Usability testing in medical informatics: cognitive approaches to evaluation of information systems and user interfaces. Proc AMIA Annu Fall Symp. 1997:218–22 Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2233486/. [cited 2021 Feb 22].

  47. U.S. General Services Administration. System Usability Scale (SUS). [place unknown]: U.S. General Services Administration; 2021. Available from: https://www.usability.gov/how-to-and-tools/methods/system-usability-scale.html. [cited 2021 Feb 22].

  48. Brooke J. SUS: a “quick and dirty” usability scale. In: Jordan PW, Thomas B, Weerdmeester BA, McClelland IL, editors. Usability evaluation in industry. London: Taylor and Francis; 1996. p. 189–94.

    Google Scholar 

  49. Ritchie J, Spencer L. Qualitative data analysis for applied policy research. In: Huberman AM, Miles MB, editors. The qualitative researcher's companion. Thousand Oaks: SAGE Publications, Inc.; 2002. p. 305–29.

    Chapter  Google Scholar 

  50. Birken SA, Rohweder CL, Powell BJ, Shea CM, Scott J, Leeman J, et al. T-CaST: an implementation theory comparison and selection tool. Implement Sci. 2018;13(1):143. https://doi.org/10.1186/s13012-018-0836-4 Available from: https://implementationscience.biomedcentral.com/articles/10.1186/s13012-018-0836-4. [cited 2021 Feb 22].

    Article  PubMed  PubMed Central  Google Scholar 

  51. Kastner M, Makarski J, Hayden L, Lai Y, Chan J, Treister V, et al. Improving KT tools and products: development and evaluation of a framework for creating optimized, knowledge-activated tools (KaT). Implement Sci Commun. 2020;1:47. https://doi.org/10.1186/s43058-020-00031-7. Available from: https://implementationsciencecomms.biomedcentral.com/articles/10.1186/s43058-020-00031-7. [cited 2021 Feb 22].

  52. Simpson J. Everyone belongs: a toolkit for applying intersectionality. Ottawa: Canadian Research Institute for the Advancement of Women; 2009. Available from: https://www.criaw-icref.ca/en/product/everyone-belongs%2D%2Da-toolkit-for-applying-intersectionality. [cited 2021 Feb 22]

    Google Scholar 

  53. Hankivsky O. Intersectionality 101. Vancouver: The Institute for Intersectionality Research & Policy, SFU; 2014. Available from: https://www.researchgate.net/profile/Olena-Hankivsky/publication/279293665_Intersectionality_101/links/56c35bda08ae602342508c7f/Intersectionality-101.pdf. [cited 2021 Feb 22]

    Google Scholar 

  54. Jull J, Giles A, Graham ID. Community-based participatory research and integrated knowledge translation: advancing the co-creation of knowledge. Implement Sci. 2017;12(1):150. https://doi.org/10.1186/s13012-017-0696-3 Available from: https://implementationscience.biomedcentral.com/articles/10.1186/s13012-017-0696-3. [cited 2021 Feb 22].

    Article  PubMed  PubMed Central  Google Scholar 

  55. Enhancing KT projects with an intersectional lens. Project limitations. Toronto: Knowledge Translation Program; c2019. Available from: https://knowledgetranslation.net/portfolios/intersectionality-and-kt/. [cited 2021 Jul 5]; [second tab].

  56. Diversity Arts Australia, The British Council. About the Creative Equity Toolkit. Parramatta: Diversity Arts Australia, The British Council. Available from: https://creativeequitytoolkit.org/about-us/. [cited 2021 Feb 22].

  57. Hunt S. Dialogue on intersectionality and indigeneity: summary of themes. Vancouver: Institute for Intersectionality Research and Policy; 2013. Available from: https://www.academia.edu/4677649/Dialogue_On_Intersectionality_and_Indigeneity_Summary_of_Themes. [cited 2021 Feb 22]

    Google Scholar 

  58. Clark N. Red intersectionality and violence-informed witnessing praxis with Indigenous girls. Girlhood Stud. 2016;9(2):46–64. https://doi.org/10.3167/ghs.2016.090205 Available from: https://www.berghahnjournals.com/view/journals/girlhood-studies/9/2/ghs090205.xml. [cited 2021 Sep 22].

    Article  Google Scholar 

Download references

Acknowledgements

We would like to thank all who have contributed to this project, including all development committee members listed here: https://knowledgetranslation.net/portfolios/intersectionality-and-kt/. We would also like to thank Tom Lee for research assistance in this project.

Funding

This study was funded by a CIHR team grant investigating the impact of gender on Knowledge Translation interventions (competition number - 201702IGK) to SES. KMS received support from the Canada Research Chairs program. IDG is a recipient of a CIHR Foundation Grant (FDN #143237). The funding bodies had no role in the design of the study; collection, analysis, interpretation of data; or in writing the manuscript.

Author information

Authors and Affiliations

Authors

Contributions

All authors contributed to the conception and design of the work, interpreted data, critically revised and approved the final manuscript. In addition, KMS led the development committee meetings and manuscript writing, DK led usability testing data collection, analysis, drafted and revised tools, SES and JEM conceived the project and obtained funding. CK and DL were members of the development committee. All members of the development committee equally contributed to reviewing the tools, though not all members are authors on this work.

Corresponding author

Correspondence to Kathryn M. Sibley.

Ethics declarations

Ethics approval and consent to participate

Ethics approval was obtained from the St. Michael’s Hospital Research Ethics Board (REB Title: Intersectionality & Knowledge Translation Interventions; REB Number: 17–273). All methods were carried out in accordance with relevant guidelines and regulations. Usability testing participants provided informed consent prior to the start of the interviews.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: Appendix A.

Development Committee Terms of Reference.

Additional file 2:  Appendix B.

Example Usability Testing Interview Guide (Intersectionality Guide).

Additional file 3: Appendix C.

Development committee discussions and decisions for tool drafts.

Additional file 4: Appendix D.

Usability testing issues for development committee discussion and decisions.

Additional file 5:

Final intersectionality and knowledge translation tools.

Additional file 6: Supplementary Table 1.

Summary of intersectionality and knowledge translation toolkit.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Sibley, K.M., Kasperavicius, D., Rodrigues, I.B. et al. Development and usability testing of tools to facilitate incorporating intersectionality in knowledge translation. BMC Health Serv Res 22, 830 (2022). https://doi.org/10.1186/s12913-022-08181-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12913-022-08181-1

Keywords

  • Health equity
  • Gender-based analysis
  • Knowledge translation
  • Integrated knowledge translation