Skip to main content

Conceptualising engagement with HIV care for people on treatment: the Indicators of HIV Care and AntiRetroviral Engagement (InCARE) Framework

Abstract

Background

As the crisis-based approach to HIV care evolves to chronic disease management, supporting ongoing engagement with HIV care is increasingly important to achieve long-term treatment success. However, ‘engagement’ is a complex concept and ambiguous definitions limit its evaluation. To guide engagement evaluation and development of interventions to improve HIV outcomes, we sought to identify critical, measurable dimensions of engagement with HIV care for people on treatment from a health service-delivery perspective.

Methods

We used a pragmatic, iterative approach to develop a framework, combining insights from researcher experience, a narrative literature review, framework mapping, expert stakeholder input and a formal scoping review of engagement measures. These inputs helped to refine the inclusion and definition of important elements of engagement behaviour that could be evaluated by the health system.

Results

The final framework presents engagement with HIV care as a dynamic behaviour that people practice rather than an individual characteristic or permanent state, so that people can be variably engaged at different points in their treatment journey. Engagement with HIV care for those on treatment is represented by three measurable dimensions: ‘retention’ (interaction with health services), ‘adherence’ (pill-taking behaviour), and ‘active self-management’ (ownership and self-management of care). Engagement is the product of wider contextual, health system and personal factors, and engagement in all dimensions facilitates successful treatment outcomes, such as virologic suppression and good health. While retention and adherence together may lead to treatment success at a particular point, this framework hypothesises that active self-management sustains treatment success over time. Thus, evaluation of all three core dimensions is crucial to realise the individual, societal and public health benefits of antiretroviral treatment programmes.

Conclusions

This framework distils a complex concept into three core, measurable dimensions critical for the maintenance of engagement. It characterises elements that the system might assess to evaluate engagement more comprehensively at individual and programmatic levels, and suggests that active self-management is an important consideration to support lifelong optimal engagement. This framework could be helpful in practice to guide the development of more nuanced interventions that improve long-term treatment success and help maintain momentum in controlling a changing epidemic.

Peer Review reports

Background

Engagement is increasingly important as the public health response to the HIV epidemic evolves

The HIV epidemic has been one of the most influential in the last century, with devastating economic, social and health consequences [1]. However, with increasing success in providing access to antiretroviral treatment (ART), HIV has evolved from an acute illness with inevitable mortality to a chronic disease that is managed by millions of patients as a normal part of their life and community [2]. As the ‘lower hanging fruit’ of access is achieved [3], the goals of ‘success’ have shifted from a focus on diagnosis and linkage to services, to long-term maintenance of treatment success for those on ART [4].

This shift in focus to long-term treatment success has changed the health system’s appreciation of the patient’s role in ART management, valuing patient ownership of their medication management and of their interactions with the health system [5]. It also puts pressure on health systems to make long-term investments in treatment and service delivery that support sustained adherence to ART [6]. There is increasing recognition of the need for health system delivery of people-centred services to support consistent, high levels of engagement as a route to maintaining treatment success [7-9]. This puts the health system’s support of optimal engagement at the “top of the list of policy priorities” [10] to facilitate the benefits of accessible treatment in controlling the epidemic [11-14] and maintain the public health gains that scaling ART has delivered [15].

Engagement is a complex concept

Long-term engagement with care is a fluctuating and dynamic process [16]. Initiating ART represents a ‘biographical disruption’[17] where daily life and its meaning, relationships, social networks and plans for the future are upset and need to be recalibrated [18]. Once on established treatment, people continue to move in and out of the system over their treatment journey [19, 20] as factors in their life shift and change [21]. The treatment journey requires navigation of a medically and socially complex condition, negotiation of often unrealistic healthcare worker expectations, coping with social and internal stigma and management of the health system’s demands for the exceptional levels of engagement needed to achieve treatment success – all while balancing their treatment with the competing priorities in their often challenging day-to-day lives [22].

Engagement with HIV care is also complicated by societal demands that are not present in non-communicable chronic diseases: sustaining virologic suppression prevents onwards transmission, making engagement with care a ‘pro-social behaviour’ and a moral obligation [22]. This places an added burden of treatment success directly on the patient in addition to their own health goals, despite myriad personal, health system and contextual factors influencing patient engagement and tipping them towards or away from engagement with care [21].

The shift in conceptualising HIV as a chronic condition over the past 20 years has been helpful to re-orient HIV services and research. However, the conventional understanding of ‘chronic disease’ as stable, manageable, asymptomatic and linked to lifestyle, fails to capture the experience of people living with HIV [6]. The available chronic disease language’s inadequacy in describing the realities of people’s relationship with HIV adds to the complexity of articulating an understanding of life-long HIV care engagement [6].

Engagement is not a well-defined concept

The term ‘engagement’ is often not explicitly defined in healthcare literature [23], and is used loosely in HIV-specific literature to refer to a broad spectrum of ‘engagements’ with the health system: linkage to care [24], interaction with or use of services [25-27], appointment attendance [28], medication and schedule adherence [29], active participation in care [30, 31] or patient management of their condition [32]. It is often used in the context of the Joint United Nations Programme on HIV/AIDS (UNAIDS) 95–95-95 targets (aim to have 95% of people living with HIV know their status, 95% of those on ART and 95% of those virologically suppressed by 2030 [33]) and the cascade of HIV care (diagnosis, linkage, retention and virologic suppression) [34].

While these concepts are interrelated [7] and often used interchangeably [35], each has different implications for intervention and strategy decisions [16]. Evaluations often focus on one specific dimension, such as retention or adherence, and so do not account for the complex, multi-dimensional, dynamic process of engagement [23]. Lack of clear definitions also hamper comparable measurement of engagement with HIV care [36], making it more difficult to compare programmes or interventions and make evidenced-based decisions [37]. This is particularly pertinent for less commonly evaluated dimensions such as active participation or self-management, which are not routinely measured. This results in little evidence on their use and a lack of data to clarify their definition and their relationship to ‘engagement’. This loop reinforcing the lack of data will require an iterative process of incremental gains in clarity, which is essential if health services are to operationalise the concept of engagement to facilitate sustained HIV treatment success [23].

To address the demands for more nuanced definitions of engagement [23] and the ambiguity around this crucial concept that limits its evaluation, we developed a pragmatic framework to describe engagement with HIV care for people on ART.

Methods

The definition of the term ‘engagement’ is often assumed to be commonly understood in literature. The clarification of such ‘taken for granted’ concepts is gaining popularity to challenge how we think about concepts to inform clinical practice and guide research [23]. We examined the concept of engagement with HIV care from the perspective of the health service’s evaluation of engagement, to guide clinical management and implementation of interventions to improve outcomes. While Morse’s Pragmatic Utility approach was not explicitly used, in line with the approach’s focus on the usefulness of a concept in practice [38] this framework was initially developed with a specific purpose: to inform the search and analysis stages of a systematic scoping study of measures of engagement with HIV care, by identifying critical, measurable dimensions of engagement from a health service-delivery perspective to guide evaluation [39]. However, it was found by stakeholders to be useful more broadly in making sense of the complex concept of engagement with HIV care and to articulate an understanding that could potentially be useful in practice.

Framework development

The process of framework development was pragmatic and iterative. Three broad rounds drew on multiple inputs: the research team’s experience, a narrative literature review and framework mapping, a process of expert stakeholder engagement and a process of identifying and categorising measures of engagement scoped from the literature in a formal scoping study (Fig. 1). The initial set of engagement dimensions was iteratively refined to produce the framework presented below.

Fig. 1
figure 1

Iterative process of refining the engagement framework and categorisation of measures of engagement

The research team had experience in clinical management of HIV patients as well as programme and health system development and evaluation for HIV and other conditions. The narrative review identified a number of definitions of engagement with HIV care such as Bright et al.’s definition of engagement [23]. It also identified existing frameworks that contributed to the process, including: the UNAIDS 95–95-95 cascade [33], the concept of the ‘revolving door’ of HIV care retention [34], Theories of Practice applied to HIV engagement [10], the Chronic Care Model [40] and the Situated Information Motivation Behavioral Skills Model of Health Care Initiation and Maintenance model [41]. The narrative review was supplemented by frameworks in related areas such as the Mutuality Framework of Engagement with Pre Exposure Prophylaxis [42] (see Supplementary material Box 1 and Fig. 1 for further information on these definitions and frameworks).

The stakeholder engagement process is described in detail in the Supplementary material (Box 2, Tables 1 and 2, and Figure 2). In short, 13 expert stakeholders (ranging from HIV clinicians to programme managers and academic researchers with experience in sub-Saharan African contexts) were consulted at multiple points in the framework development process to provide input on: the initial assumptions and focus, the framing of engagement with care, distinction between concepts that reflect engagement itself from those that reflect factors influencing it, and the applicability of the framework in practice.

Application of the framework to the search for and categorisation of measures of engagement in a parallel scoping study process, identified gaps in the description of engagement and overlaps in the dimensions. This directed further literature review and stakeholder input. The framework was tested by applying it to a list of 110 measures of engagement identified in the related scoping study to categorise them into mutually exclusive dimensions of engagement with HIV Care [39].

The assumptions and decisions made in developing the initial working definition of engagement are outlined in Table 1.

Table 1 Assumptions on engagement and their impact on the development of the initial framework

Results

Proposed definition of engagement

Building on Bright et al.’s definition of engagement [23], the following definition is proposed as a definition of engagement with HIV care for those on ART, from the health service-delivery perspective:

Engagement is a dynamic state comprising three dimensions: retention in services, adherence to medication and active self-management of care. It incorporates a process of connecting with all aspects of HIV care (services and treatment itself), interacting with the individual as an active, invested collaborator in healthcare and facilitating maintenance of successful treatment outcomes.

Indicators of HIV care and antiretroviral engagement: the ‘InCARE’ framework

The developed framework is shown in Fig. 2. It describes engagement for people already on ART as separate from engagement with services to initiate treatment [41]. Engagement with HIV care is a multi-dimensional set of observable behaviours that people practice, rather than a static characteristic of the individual themselves as being an ‘engaged’ or ‘good’ patient. Engagement behaviour is dynamic rather than a permanent state, with people practising variable engagement at different points over the lifelong course of their treatment journey. Engagement with HIV care is reflected by measurable actions that reflect whether a patient is consistently engaging with services and treatment: ‘retention’, ‘adherence’ and ‘active self-management’.

Fig. 2
figure 2

Indicators of HIV Care and Antiretroviral Engagement (InCARE) Framework, drawing from multiple definitions of retention [44-46], adherence [47-49], active self-management [50-52] and treatment outcomes [53-55]

The practice of engagement is separate from the wider contextual, health system and individual factors that are important for engagement, but influence this practice rather than being considered engagement itself [21, 56, 57]. Contextual influences include community dynamics, social resources, socioeconomic status, responsibilities, stigma, social norms, social capital and support, societal demands and politics [21, 22, 58]. Health system organisation covers delivery systems and differentiated model options, supply of medication, health information systems, decision support and relationships with healthcare workers [21, 59]. Individual factors influencing the practice of engagement behaviour include information, motivation and behavioural skills, self-efficacy, acceptance of diagnosis, demographics, clinical state (such as CD4 count) and comorbidities (including mental health and substance abuse amongst others) [10, 21, 41].

Maintenance of engagement with HIV care facilitates and sustains successful treatment outcomes. Treatment success is the ultimate goal of HIV programmes [60, 61], as it facilitates the benefits of ART: individual health benefits of improved health and reduced early mortality [62]; public health benefits including reduced horizontal and vertical transmission [7, 60], reduced costs associated with managing antiretroviral resistant cases and clinically unwell patients [63] and the health system advantages of stable patients currently qualifying for differentiated service delivery models [63]; and societal benefits such as a healthier working age population available to parent their children [1].

Discussion

Clinical and public health approaches have shifted from treating HIV as an acute emergency to viewing it as a chronic disease: where the patient takes the primary role in the management of their condition and differentiated service delivery models reduce contact with both facilities and providers [5, 53]. This makes maintaining engagement with HIV care more crucial to sustain treatment success [7, 8] and highlights the importance of person-centred support for a person’s lifelong engagement with HIV care [7-9]. We developed a pragmatic framework to conceptualise engagement for people on ART for use by health services to comprehensively evaluate engagement, understand programmes and target interventions to optimise engagement with HIV care.

Pragmatic utility of the InCARE framework for research, implementing programmes and clinical management

Engagement as a behaviour rather than a characteristic of individuals

This framework recognises that maintaining engagement with care is an observable behaviour, distinct from the engagement required to initiate ART [41]. Understanding engagement as a behaviour, rather than a static patient phenotype, shifts the focus away from a dichotomous ‘good’ or ‘bad’ patient categorisation. This normalises the cycle of engagement as people disengage and re-engage with care over time [19, 20]. Suboptimal engagement can then be understood as an expected reality and managed with compassion rather than punished as a failure. This also shifts the health service’s focus to a person-centred approach, facilitating the treatment of patients as active participants in their treatment rather than passive recipients of care who need a paternalistic relationship with the system to enforce engagement [31, 64].

Engagement as a dynamic process rather than a linear cascade

The cascade approach to HIV engagement has served as a framework for identifying gaps in coverage and service performance [44]. It has contributed to achieving some of the ‘low hanging fruit’ in the progress of HIV services such as improved standardisation of measures of retention and scale up of ART [10]. However, it has been criticised as overly linear, unidirectional, and for not accounting for the lived realities of the complex patient journey where patients move in and out of care over time [19, 20, 65, 66]. Ehrenkranz et al. proposed a more cyclical cascade to evaluate the retention dimension at a population level more accurately by structuring the cascade to reflect “actual … behaviour” and making the concept of the cycle of disengagement and re-engagement explicit and expected [34]. This approach is well-suited to monitoring national and global programmes [67].

The InCARE framework attempts to develop this further and describes engagement as a dynamic process where patients move in and out of the desired ‘fully engaged’ state [68] depending on their fulfilment of the three dimensions of retention, adherence and active self-management. This conceptualisation focuses on the relationship between elements of the behaviour to complement the cascade approach, which describes the timing and sequence of transitions between states along a cascade. This again normalises the cycle of engagement [19, 20], and helps to direct interventions to periods of time when individuals are require additional support. It also normalises that individuals will be sub-optimally engaged at some point in their treatment journey, and highlights the need for continued monitoring of engagement once people are stable on ART.

Engagement as a mediator of influential factors on treatment outcomes

Engagement behaviour is an observable part of a complex process influenced by many layers within and outside of a person’s control: engagement is negotiated between the patient, their experience of HIV and the health system, and their context of competing priorities [10, 41]. The InCARE framework places engagement as distinct from influential individual, contextual and health system factors that influence this behaviour [69]. The framework also distinguishes between engagement behaviour and treatment outcomes such as virologic suppression or health status [54]. This distinction helps to disentangle the effect that changes in influential factors (such as addressing mental health issues, reducing community stigma or extending clinic opening hours) have on the dimensions of engagement, on engagement overall and ultimately on treatment success.

The scale up of ART has resulted in massive public health gains [15], but has shifted the gaps from treatment access to engagement as the key modifier of the success [4] – a much more complicated outcome to achieve. As incremental gains in treatment success become harder to achieve, it becomes more difficult to show efficacy of interventions if only viral outcomes are considered [70]. Recognising engagement and its dimensions as a measurable state on the path to treatment success [4] could help to evaluate the impact of interventions more accurately and develop more nuanced services targeted to leverage a person’s strengths and to meet their needs. It could also help to identify influential factors to intervene on and understand how they ultimately result in improvements in treatment success.

Engagement as a comprehensive set of measurable dimensions

The InCARE framework describes engagement as comprised of three measurable dimensions that reflect interactions with both the services and treatment itself: retention, adherence and active self-management. To achieve long-term treatment success, a person must interact with health services to receive ART (retention), take their treatment according to the schedule prescribed by the clinician so that it can work optimally (adherence) and be actively committed to managing their health between infrequent facility visits (active self-management). The InCARE framework situates each dimension as necessary, but not on its own sufficient, to sustain treatment success over time.

While the concepts of retention and adherence are correlated, the InCARE framework places them as distinct dimensions within engagement behaviour, with retention reflecting periodic engagement with healthcare services and adherence reflecting daily engagement with treatment itself. People can be in one state without necessarily being in the other [71]: for example, people are commonly considered ‘retained’ (medication was dispensed and a viral load measured) but not ‘fully adherent’ (viral load is unsuppressed). Conversely, patients may struggle with barriers to attending appointments but manage their daily adherence well [21]. Suboptimal retention or adherence reflect different issues and require different approaches to improve treatment outcomes, thus the differentiation is important.

Treatment success needs to be sustained over a lifetime of ART to prevent drug resistance and facilitate the benefits that ART provides [72, 73]  While retention and adherence may be sufficient to result in treatment success at a particular moment in a person’s treatment journey (reflected for example by a suppressed viral load), both dimensions of engagement are vulnerable to the many shifting individual, contextual and health system factors (and the dynamics between them [69]) which combine to tip people towards or away from optimal engagement with care [21]. Retention and adherence have also been criticised for not fully defining the multi-dimensional “care engagement process” [74, 75].

Maintenance of engagement, and subsequent sustained treatment success, includes an element of ownership, active involvement and self-management of care between facility visits – termed ‘active self-management’ in the InCARE framework. Along with retention and adherence, self-management is seen as essential in lifelong treatment [40], as the health system’s role in achieving ART success is limited to a few interactions with a person over the course of a year and individuals are ultimately responsible for the lifetime task of day-to-day management of their condition (whether they are managing it well or not, they are in fact managing their health) [50]. People are increasingly required to take an active role in their care: to use medication properly, interpret and report symptoms correctly, make decisions on when to seek care, adjust to the new reality of living with HIV (and what this means for them socially and economically), cope with the emotional consequences of their disease, participate in treatment decisions and manage their care to prevent onward transmission [2]. Fully engaged patients are “informed, activated patients in partnership with their physicians”, who can cope with successes, setbacks and living with uncertainty to remain engaged [2].

Retention, adherence and viral suppression measures detect issues late in the disengagement process [74], so examining active self-management may identify patients with a different risk of ‘tipping’ towards poor engagement and so facilitate earlier intervention to maintain treatment success and prevent poor outcomes [13, 76]. Active self-management could also provide some measure of resilience to changing external factors, so that achieving ‘full engagement’ requires more than simply having (retention) and taking (adherence) a pill every day. It requires all three InCARE dimensions, making the case for the important role that active self-management plays in lifelong engagement with HIV care.

Each dimension may have differing relevance for different use cases: programme and population monitoring, research to develop a deeper understanding of how and why patients engage with their treatment to build better services and directing the clinical management of individual patients. Operationalising this understanding of engagement starts with identifying how each dimension is measured for each of these use cases. Conceptualising engagement as three measurable dimensions allows metrics to be categorised by the dimension they evaluate. This could help to disentangle the effect of interventions more granularly than only considering treatment outcomes [70], particularly if the intervention contributes to better engagement in one dimension but is not sufficient to help the patient sustain treatment success.

HIV is a relatively well-researched disease, and despite its devastating effects has had a transformational effect on the ability of health systems in sub-Saharan Africa to manage chronic disease [77]. Many of the health system barriers that reduce service support of engagement for non-communicable disease patients are common to those living with HIV [78]. Retention and adherence strategies in particular are an aspect of HIV care that has been successful in improving outcomes and could be leveraged to improve the care of many other less well-funded chronic diseases, such as hypertension and diabetes, in lower-resource settings [77, 78]. The dimensions described in this manuscript could be translated and adapted to explore engagement with non-communicable chronic disease, supporting research, programme evaluation as well as the development and implementation of interventions to improve engagement with care.

Strengths and limitations

This framework was developed in an iterative process between defining the dimensions of engagement and categorising measures of engagement with HIV care scoped from the literature [39]. It drew on multiple inputs from literature as well as experienced clinicians and researchers to articulate the synthesis of a more comprehensive understanding of engagement. Expert stakeholders contributed to delineating the dimensions for greater clarity, separating factors that affect engagement (such as self-efficacy and motivation) from engagement behaviour itself, teasing out the active self-management dimension and finding terms to describe this component and the longitudinal nature of the engagement process more clearly. Pragmatism was also a major concern: how framing engagement differently relates to a change in practice was kept at the forefront of discussions.

A systematic process was used to develop this framework, but it may have benefitted from a guiding approach, such as a pragmatic utility concept clarification analysis. It has also not yet resulted in a definitive definition nor an incontrovertible framework, but is rather an incremental step towards greater clarification of the concept. The InCARE framework offers the next step in the exploration of engagement and its importance in individual care and public health. Further work is required to clarify engagement’s role in the facilitation of treatment success and to build evidence on the active self-management dimension.

While retention (measured by interaction with the health system) and adherence (measured by pharmacy refills, antiretroviral concentrations, laboratory results, healthcare worker assessment, pill counts or self-report) are commonly evaluated aspects of engagement behaviour [74], the science of self-management and related concepts is still at an early stage [79] and has not been prioritised in lower-resource settings [72]. Thus, it is not routinely measured and so is not as well understood as retention or adherence. The dimension of active self-management also required the most refinement (with input from expert stakeholders and the results of the scoping study [39]), with concepts like motivation and self-efficacy being removed from this dimension and placed under ‘individual influential factors’ as an ability to engage rather than a reflection of engagement itself. This framework places active-management in a prominent role as one of the three core dimensions of engagement behaviour, but this position is dependent on further work to identify comparable measures of active self-management and address the paucity of evidence of its association with retention, adherence and the maintenance of treatment success.

Conclusions

Engagement with HIV care is a critical but complex concept. Supporting people to maintain HIV engagement across a lifetime (with so many factors outside the health system’s control) requires more comprehensive health system approaches than those aimed at only providing access to treatment. The programmes that were so successful in scaling up access to treatment now face a new challenge in adapting their response to the evolving needs of people on ART: developing more person-centred interventions rather than continuing to implement the existing siloed, vertical programmes that work for most people most of the time, but do not accommodate the realities people face in managing their treatment within their specific context [10, 37].

The InCARE framework offers an incremental step towards a more comprehensive and standardised understanding of engagement as a dynamic behaviour. It distils the complex concept of engagement into three measurable dimensions that are pragmatically applicable to both the development of more nuanced interventions, and to characterising elements that the health system could measure for more complete evaluation of engagement at individual and population levels. This framework suggests that active self-management is an important consideration in understanding the maintenance of engagement with HIV care, but further work is needed to build evidence for its importance. As the demands on HIV services become more complex, more thorough understanding and evaluation of engagement with care will be required to support sustained successful treatment outcomes over a lifetime of HIV interactions, and this framework could be useful to advocate for such an approach.

Availability of data and materials

Data is included as supplementary material. Further data requests can be directed to the corresponding author.

Abbreviations

ART:

Antiretroviral treatment

HIV:

Human Immunodeficiency Virus

UNAIDS:

United Nations Programme on HIV/AIDS

References

  1. Barnett T, Whiteside A, Desmond C. The social and economic impact of HIV/AIDS in poor countries: a review of studies and lessons. Prog Dev Stud. 2001;1(2):151–70.

    Article  Google Scholar 

  2. Swendeman D, Ingram BL, Rotheram-Borus MJ. Common elements in self-management of HIV and other chronic illnesses: an integrative framework. AIDS Care. 2009;21(10):1321–34.

    Article  PubMed  PubMed Central  Google Scholar 

  3. McKinney O, Modeste NN, Lee JW, Gleason PC. Predicting Malawian women’s intention to adhere to antiretroviral therapy. J Public Health Res. 2015;4(2):533.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Sikazwe I, Eshun-Wilsonid I, Sikombe K, Czaicki N, Somwe P, Mody A, et al. Retention and viral suppression in a cohort of HIV patients on antiretroviral therapy in Zambia: Regionally representative estimates using a multistage-sampling-based approach. PLoS Med. 2019;16(5):1–17.

    Article  Google Scholar 

  5. Angwenyi V, Aantjes C, Kajumi M, De Man J, Criel B, Bunders-Aelen J. Patients experiences of self-management and strategies for dealing with chronic conditions in rural Malawi. PLoS ONE. 2018;13(7): e0199977.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Colvin CJ. HIV/AIDS, chronic diseases and globalisation. Global Health. 2011;7:1–6.

    Google Scholar 

  7. Mugavero MJ, Davila JA, Nevin CR, Giordano TP. From access to engagement: measuring retention in outpatient HIV clinical care. AIDS Patient Care STDS. 2010;24(10):607–13.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Wachira J, Middlestadt S, Reece M, Peng CYJ, Braitstein P. Physician communication behaviors from the perspective of adult HIV patients in Kenya. Int J Qual Health Care. 2014;26(2):190–7.

    Article  PubMed  Google Scholar 

  9. Geng EH, Odeny TA, Lyamuya R, Nakiwogga-Muwanga A, Diero L, Bwana M, et al. Retention in care and patient-reported reasons for undocumented transfer or stopping care among HIV-infected patients on antiretroviral therapy in Eastern Africa: application of a sampling-based approach. Clin Infect Dis. 2016;62(7):935–44.

    Article  PubMed  Google Scholar 

  10. Skovdal M, Wringe A, Seeley J, Renju J, Paparini S, Wamoyi J, et al. Using theories of practice to understand HIV-positive persons varied engagement with HIV services: a qualitative study in six Sub-Saharan African countries. Sex Transm Infect. 2017;93(Supplement 3):52977.

    Google Scholar 

  11. Ware NC, Wyatt MA, Geng EH, Kaaya SF, Agbaji OO, Muyindike WR, et al. Toward an understanding of disengagement from HIV treatment and care in Sub-Saharan Africa: a qualitative study. PLoS Med. 2013;10(1):e1001369.

    Article  PubMed  PubMed Central  Google Scholar 

  12. Nnambalirwa M, Govathson C, Evans D, McNamara L, Maskew M, Nyasulu P. Markers of poor adherence among adults with HIV attending Themba Lethu HIV Clinic, Helen Joseph Hospital, Johannesburg, South Africa. Trans R Soc Trop Med Hyg. 2016;110(12):696–704. Available from: http://trstmh.oxfordjournals.org/content/by/year/2013

  13. Mbengue MAS, Chasela C, Onoya D, Mboup S, Fox MP, Evans D. Clinical predictor score to identify patients at risk of poor viral load suppression at six months on antiretroviral therapy: results from a prospective cohort study in Johannesburg South Africa. Clin Epidemiol. 2019;11:359–73.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Adefolalu A, Nkosi Z, Olorunju S, Masemola P. Self-efficacy, medication beliefs and adherence to antiretroviral therapy by patients attending a health facility in Pretoria. South Afr Fam Pract. 2014;56(5):281–5.

    Article  Google Scholar 

  15. Fox MP, Bor J, Brennan AT, MacLeod WB, Maskew M, Stevens WS, et al. Estimating retention in HIV care accounting for patient transfers: a national laboratory cohort study in South Africa. PLoS Med. 2018;15(6): e1002589.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Font H, Rollins N, Essajee S, Becquet R, Foster G, Mangwiro AZ, et al. Retention-in-care in the PMTCT cascade: definitions matter! Analyses from the INSPIRE projects in Malawi, Nigeria and Zimbabwe. J Int AIDS Soc. 2020;23(10): e25609.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Bruton J, Rai T, Day S, Ward H. Patient perspectives on the HIV continuum of care in London: a qualitative study of people diagnosed between 1986 and 2014. BMJ Open. 2018;8(3):e020208.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Bury M. Chronic illness as biographical disruption. Sociol Health Illn. 1982;4(2):167–82.

    Article  CAS  PubMed  Google Scholar 

  19. Kaplan SR, Oosthuizen C, Stinson K, Little F, Euvrard J, Schomaker M, et al. Contemporary disengagement from antiretroviral therapy in Khayelitsha, South Africa: a cohort study. PLoS Med. 2017;14(11):1–24.

    Article  Google Scholar 

  20. Osler M, Hilderbrand K, Goemaere E, Ford N, Smith M, Meintjes G, et al. The continuing burden of advanced HIV disease over 10 years of increasing antiretroviral therapy coverage in South Africa. Clin Infect Dis. 2018;66(Figure 1):S118-25.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Eshun-Wilson I, Rohwer A, Hendricks L, Oliver S, Garner P. Being HIV positive and staying on antiretroviral therapy in Africa : A qualitative systematic review and theoretical model. PlosONE. 2019;14(1):e0210408. https://doi.org/10.1371/journal.pone.0210408&type=printable.

    Article  CAS  Google Scholar 

  22. Paparini S, Rhodes T. The biopolitics of engagement and the HIV cascade of care: a synthesis of the literature on patient citizenship and antiretroviral therapy. Crit Public Health. 2016;26(5):501–17.

    Article  Google Scholar 

  23. Bright FAS, Kayes NM, Worrall L, McPherson KM. A conceptual review of engagement in healthcare and rehabilitation. Disabil Rehabil. 2015;37(8):643–54.

    Article  PubMed  Google Scholar 

  24. Sikkema KJ, Mulawa MI, Robertson C, Watt MH, Ciya N, Stein DJ, et al. Improving AIDS Care After Trauma (ImpACT): pilot outcomes of a coping intervention among HIV-infected women with sexual trauma in South Africa. AIDS Behav. 2018;22(3):1039–52.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Lee H, Genberg BL, Nyambura M, Hogan J, Braitstein P, Sang E. State-Space Models for Engagement, Retention, and Reentry in the HIV Care Cascade. In: Conference on Retroviruses and Opportunistic Infections (CROI). Boston: CROI; 2016.

  26. Reidy W, Agarwal M, Lamb M, Hawken M, Chege D, Elul B, et al. Loss to follow-up: determining outcomes for adults enrolled in HIV services in Kenya. Conference on Retroviruses and Opportunistic Infections (CROI). 2014;22(E-1):564. Available from: http://ovidsp.ovid.com/ovidweb.cgi?T=JS&PAGE=reference&D=emed15&NEWS=N&AN=75008523

  27. Phillips T, Zerbe A, Ronan A, Hsiao NYM, Petro G, McIntyre JA, et al. Measures of engagement in routine HIV care to predict elevated viral load on ART. In: Conference on Retroviruses and Opportunistic Infections (CROI). Seattle: CROI; 2017.

  28. Howarth A, Apea V, Michie S, Morris S, Sachikonye M, Mercer C, et al. REACH: a mixed-methods study to investigate the measurement, prediction and improvement of retention and engagement in outpatient HIV care. Health Serv Deliv Res. 2017;5(13):1–160.

    Article  Google Scholar 

  29. Farrell L, Ingersoll K, Ceperich SD. Enhancing patient adherence: promoting engagement via positive patient-provider relationships in HIV/AIDS care. Med Encount. 2009;23(2):69–71.

    PubMed  PubMed Central  Google Scholar 

  30. Maclachlan EW, Shepard-Perry MG, Ingo P, Uusiku J, Mushimba R, Simwanza R, et al. Evaluating the effectiveness of patient education and empowerment to improve patient-provider interactions in antiretroviral therapy clinics in Namibia. AIDS Care. 2016;28(5):620–7.

    Article  PubMed  Google Scholar 

  31. Chidrawi HC, Greeff M, Temane QM. Health behaviour change of people living with HIV after a comprehensive community-based HIV stigma reduction intervention in North-West Province in South Africa. SAHARA J. 2014;11(1):222–32.

    Article  PubMed  Google Scholar 

  32. Areri H, Marshall A, Harvey G. Factors influencing self-management of adults living with HIV on antiretroviral therapy in Northwest Ethiopia: a cross-sectional study. BMC Infect Dis. 2020;20(1):879.

    Article  PubMed  PubMed Central  Google Scholar 

  33. UNAIDS. Understanding Fast-Track Targets. Accelerating action to end the AIDS epidemic by 2030. UNAIDS Reference. 2015;1-12. https://www.unaids.org/sites/default/files/media_asset/201506_JC2743_Understanding_FastTrack_en.pdf.

  34. Ehrenkranz P, Rosen S, Boulle A, Eaton JW, Ford N, Fox MP, et al. The revolving door of HIV care: revising the service delivery cascade to achieve the UNAIDS 95–95-95 goals. PLoS Med. 2021;18(5):1–10.

    Article  Google Scholar 

  35. Nawar E. Longitudinal analysis of interruptions in HIV care and treatment among HIV-positive pregnant women engaged in clinical care in Kigali, Rwanda. Dissertation abstracts international section a: Hum Soc Sci 2020;81(7):No-Specified.

  36. Mugglin C, Kläger D, Gueler A, Vanobberghen F, Rice B, et al. The HIV care cascade in sub-Saharan Africa: systematic review of published criteria and definitions. J Int AIDS Soc. 2021;24(7):1–14.

    Article  Google Scholar 

  37. Ford N, Eshun-wilson I, Ameyan W, Newman M, Vojnov L, Doherty M, et al. Future directions for HIV service delivery research: Research gaps identified through WHO guideline development process. PLoS Med. 2021;18(9): e1003812.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  38. Morse JM. Concept clarification: the use of pragmatic utility. In: analyzing and conceptualizing the theoretical foundations of nursing. New York, USA: Springer Publishing Company, Inc.; 2017. p. 337–53.

    Book  Google Scholar 

  39. Keene CM, Ragunathan A, Euvrard J, English M, Mcknight J, Orrell C, et al. Measuring patient engagement with HIV care in sub-Saharan Africa: a scoping study. J Int AIDS Soc. 2022;2022:26025. https://doi.org/10.1002/jia2.26025/full|10.1002/jia2.26025.

    Article  Google Scholar 

  40. MacColl Center for Health Care Innovation. The chronic care model. Improving chronic care. 2021 Cited 2021 Sep 6. Available from: http://www.improvingchroniccare.org/index.php?p=the_chronic_caremodel&s=2

  41. Rivet AK. A situated-information motivation behavioral skills model of care initiation and maintenance (sIMB-CIM): An IMB model based approach to understanding and intervening in engagement in care for chronic medical conditions. J Health Psychol. 2011;16(7):1071–81.

    Article  Google Scholar 

  42. Amico KR, Wallace M, Bekker LG, Roux S, Atujuna M, Sebastian E, et al. Experiences with HPTN 067/ADAPT study-provided open-label PrEP among women in Cape Town: facilitators and barriers within a mutuality framework. AIDS Behav. 2017;21(5):1361–75.

    Article  PubMed  Google Scholar 

  43. UNICEF. Paediatric care and treatment [Internet]. Data: HIV/AIDS. New York: UNICEF; 2021. Cited 2022 Feb 2. Available from: https://data.unicef.org/topic/hivaids/paediatric-treatment-and-care/#:~:text=Globally%2C children under age 15, those most vulnerable to HIV.

  44. Rollins NC, Becquet R, Orne-Gliemann J, Phiri S, Hayashi C, Baller A, et al. Defining and analyzing retention-in-care among pregnant and breastfeeding HIV-infected women: Unpacking the data to interpret and improve PMTCT outcomes. J Acquir Immune Defic Syndr. 1988;2014(67):S150–6.

    Google Scholar 

  45. Rosen S, Fox MP, Gill CJ. Patient retention in antiretroviral therapy programs in sub-Saharan Africa: a systematic review. PLoS Med. 2007;4(10):1691–701.

    Article  Google Scholar 

  46. Phillips TK, Orrell C, Brittain K, Zerbe A, Abrams EJ, Myer L. Measuring retention in HIV care: the impact of data sources and definitions using routine data. AIDS. 2020;34(5):749–59.

    Article  PubMed  Google Scholar 

  47. World Health Organization. Adherence to long-term therapies: Evidence for action. World Health Organization. Geneva; 2003. Available from: http://apps.who.int/iris/bitstream/%0A10665/42682/1/9241545992.pdf%0A

  48. Corless IB, Hoyt AJ, Tyer-Viola L, Sefcik E, Kemppainen J, Holzemer WL, et al. 90–90–90-Plus: Maintaining adherence to antiretroviral therapies. AIDS Patient Care STDS. 2017;31(5):227–36. Available from: http://search.ebscohost.com/login.aspx?direct=true&db=cin20&AN=123086853&site=ehost-live&authtype=ip,uid

  49. Vrijens B, de Geest S, Hughes DA, Przemyslaw K, Demonceau J, Ruppar T, et al. A new taxonomy for describing and defining adherence to medications. Br J Clin Pharmacol. 2012;73(5):691–705.

    Article  PubMed  PubMed Central  Google Scholar 

  50. Lorig KR, Holman HR, Med AB. Self-management education: history, definition, outcomes, and mechanisms. Ann Behav Med. 2003;26(1):1–7.

    Article  PubMed  Google Scholar 

  51. Angwenyi V, Bunders-Aelen J, Criel B, Lazarus JV, Aantjes C. An evaluation of self-management outcomes among chronic care patients in community home-based care programmes in rural Malawi: A 12-month follow-up study. Health Soc Care Community. 2020

  52. World Health Organization. Innovative Care for Chronic Conditions: Building Blocks for Action. Geneva: World Health Organization. Geneva, Switzerland; 2002.

    Google Scholar 

  53. Agaba PA, Genberg BL, Sagay AS, Agbaji OO, Meloni ST, Dadem NY, et al. Retention in differentiated care: multiple measures analysis for a decentralized HIV care and treatment program in North Central Nigeria. J AIDS Clin Res. 2018;9(2):756.

    Article  PubMed  PubMed Central  Google Scholar 

  54. Lazarus JV, Safreed-Harmon K, Kamarulzaman A, Anderson J, Leite RB, Behrens G, et al. Consensus statement on the role of health systems in advancing the long-term well-being of people living with HIV. Nat Commun. 2021;12(1):3–7. https://doi.org/10.1038/s41467-021-24673-w.

    Article  CAS  Google Scholar 

  55. Hine P, Smith R, Eshun‐Wilson I, Orrell C, Cohen K, Leeflang MMG, et al. Measures of antiretroviral adherence for detecting viral non‐suppression in people living with HIV. Cochrane Data Syst Rev 2018;(7). https://doi.org/10.1002/14651858.CD013080

  56. Dahlgren G, Whitehead M. Policies and strategies to promote social equity in health. Background document to WHO - Strategy paper for Europe. 1991.

  57. Blue S, Shove E, Carmona C, Kelly MP, Blue S, Shove E, et al. Theories of practice and public health: understanding ( un ) healthy practices. Crit Public Health. 2016;26(1):36–50.

    Article  Google Scholar 

  58. Topp SM, Mwamba C, Sharma A, Mukamba N, Beres LK, Geng E, et al. Rethinking retention: mapping interactions between multiple factors that influence long-term engagement in HIV care. PLoS One. 2018;13(3):e0193641. https://doi.org/10.1371/journal.pone.0193641&type=printable.

    Article  PubMed  PubMed Central  Google Scholar 

  59. Wagner EH. Organizing care for patients with chronic illness revisited. Milbank Q. 2019;97(3):659–64.

    Article  PubMed  PubMed Central  Google Scholar 

  60. Mugavero MJ, Westfall AO, Zinski A, Drainoni Mlynn, Gardner LI, Keruly JC, et al. Measuring retention in HIV care: the elusive gold standard. J Acquir Immune Defic Syndr. 2012;61(5):574–80.

    Article  PubMed  PubMed Central  Google Scholar 

  61. VandenBerg JJ, Neilands TB, Johnson MO, Chen B, Saberi P. Using path analysis to evaluate the healthcare empowerment model among persons living with HIV for antiretroviral therapy adherence. AIDS Patient Care STDS. 2016;30(11):497–505.

    Article  Google Scholar 

  62. Danforth K, Granich R, Wiedeman D, Baxi S, Padian N. Global Mortality and Morbidity of HIV/AIDS. In: Holmes KK, Bertozzi S, Bloom BR, Jha P, editors. Disease control priorities: Major Infectious Diseases. Third edit Washington DC, USA: World Bank Group; 2017. p. 29–44.

    Google Scholar 

  63. International AIDS Society. Differentiated service delivery: about. Differentiated service delivery. 2020 Cited 2020 Dec 10. Available from: https://www.differentiatedcare.org/about

  64. Richard AA, Shea K. Delineation of self-care and associated concepts. J Nurs Scholarsh. 2011;43(3):255–64.

    PubMed  Google Scholar 

  65. Hogg RS. Understanding the HIV care continuum. Lancet HIV. 2018;5:e269–70.

    Article  PubMed  Google Scholar 

  66. Mugavero MJ, Amico KR, Horn T, Thompson MA. The state of engagement in HIV care in the United States: from cascade to continuum to control. Clin Infect Dis. 2013;57(8):1164–71.

    Article  PubMed  Google Scholar 

  67. World Health Organization. Cascade data use manual: to identify gaps in HIV and health services for programme improvement. Geneva: HIV Strategic Information for Impact; 2018.

    Google Scholar 

  68. Beres LK, Schwartz S, Simbeza S, McGready J, Eshun-Wilson I, Mwamba C, et al. Patterns and predictors of incident return to HIV care among traced, disengaged patients in Zambia: analysis of a prospective cohort. J Acquir Immune Defic Syndr. 2021;86(3):313–22.

    Article  PubMed  Google Scholar 

  69. Gardner EM, McLees MP, Steiner JF, Del Rio C, Burman WJ. The spectrum of engagement in HIV care and its relevance to test-and-treat strategies for prevention of HIV infection. Clin Infect Dis. 2011;52(6):793–800.

    Article  PubMed  PubMed Central  Google Scholar 

  70. Poles G, Li M, Siril H, Mhalu A, Hawkins C, Kaaya S, et al. Factors associated with different patterns of nonadherence to HIV care in Dar es Salaam, Tanzania. J Int Assoc Provid AIDS Care. 2014;13(1):78–84.

    Article  PubMed  Google Scholar 

  71. Johnson MO, Neilands TB, Koester KA, Wood T, Sauceda JA, Dilworth SE, et al. Detecting disengagement from HIV care before it’s too late: development and preliminary validation of a novel index of engagement in HIV care. J Acquir Immune Defic Syndr (1998). 2019;81(2):145–52.

    Article  Google Scholar 

  72. Ford N, Meintjes G, Calmy A, Bygrave H, Migone C, Vitoria M, et al. Managing advanced HIV disease in a public health approach. Clin Infect Dis. 2018;66(Suppl 2):S106–10.

    Article  PubMed  PubMed Central  Google Scholar 

  73. Long L, Fox M, Sanne I, Rosen S. The high cost of second-line antiretroviral therapy for HIV/AIDS in South Africa. AIDS. 2010;24(6):915–9.

    Article  PubMed  Google Scholar 

  74. Johnson MO. The shifting landscape of health care: Toward a model of health care empowerment. Am J Public Health. 2011;101(2):265–70.

    Article  PubMed  PubMed Central  Google Scholar 

  75. Phillips T, Brittain K, Mellins CA, Zerbe A, Remien RH, Abrams EJ, et al. A self-reported adherence measure to screen for elevated HIV viral load in pregnant and postpartum women on antiretroviral therapy. AIDS Behav. 2017;21(2):450–61.

    Article  PubMed  PubMed Central  Google Scholar 

  76. Gupta N, Bukhman G. Leveraging the lessons learned from HIV/AIDS for coordinated chronic care delivery in resource-poor settings. Healthcare. 2015;3(4):215–20. https://doi.org/10.1016/j.hjdsi.2015.09.006.

    Article  PubMed  Google Scholar 

  77. Rabkin M, El-Sadr WM. Why reinvent the wheel? leveraging the lessons of HIV scale-up to confront non-communicable diseases. Glob Public Health. 2011;6(3):247–56.

    Article  PubMed  Google Scholar 

  78. Crowley T, Van der Merwe A, Kidd M, Skinner D. Measuring adolescent HIV self-management: an instrument development study. AIDS Behav. 2020;24(2):592–606.

    Article  PubMed  Google Scholar 

  79. Crowley T, van der Merwe A, Kidd M, Skinner D. Adolescent human immunodeficiency virus self-management: associations with treatment adherence, viral suppression, sexual risk behaviours and health-related quality of life. South Afr J HIV Med. 2020;21(1):1–11.

    Article  Google Scholar 

Download references

Acknowledgements

We would like to acknowledge the individual members of the InCARE Stakeholder Group, and their contribution to shaping and refining this framework: Anna Grimsrud, Beth Harley, Erin von der Heyden, Ingrid Eshun-Wilson, Ingrid Katz, Jonathan Euvrard, Kirsten Arendse, Laura Beres, Michael Mugavero, K Rivet Amico, Tali Cassidy, and Tamsin Phillips. We would also like to thank Sydney Rosen for her engagement on this topic.

Anna Grimsrud: Lead Technical Advisor, International AIDS Society

Beth Harley: HIV, AIDS, sexually transmitted infections and tuberculosis (HAST) medical officer for City of Cape Town, South Africa

Erin von der Heyden: HIV, AIDS, sexually transmitted infections and tuberculosis (HAST) medical officer for the Khayelitsha and Eastern Sub-structure of the Western Cape Department of Health, South Africa

Ingrid Eshun-Wilson: Family physician and epidemiologist, School of Medicine, Washington University, United States of America and University of Stellenbosch, South Africa

Ingrid Katz: Physician and Associate Faculty Director, Harvard Global Health Institute, Harvard University, United States of America Jonathan Euvrard: Epidemiologist at the Centre for Infectious Disease Epidemiology and Research (CIDER) and PhD student at the University of Cape Town, South Africa

Kirsten Arendse: HIV Medical Research Manager for the ‘Welcome Service’ (A differentiated service delivery model for patients who interrupt ART) for Médecins Sans Frontières’ Khayelitsha project, South Africa

Laura Beres: Assistant Scientist at Johns Hopkins Bloomberg School of Public Health, Department of International Health, Johns Hopkins, United States of America

Michael Mugavero: Director for the Center for Outcomes Effectiveness Research and Education (COERE), Co-Director of the Center For AIDS Research, and PI of an AHRQ T32 training grant in health services and outcomes research, University of Alabama at Birmingham, United States of America

K Rivet Amico: Associate Professor, Health Behaviour and Health Education, School of Public Health, University of Michigan, United States of America

Tali Cassidy: HIV Epidemiologist for Médecins Sans Frontières’ Khayelitsha project and Division of public health medicine, School of Public Health and Family Medicine, University of Cape Town, South Africa

Tamsin Phillips: Senior Lecturer in Division of Epidemiology & Biostatistics, School of Public Health & Family Medicine, University of Cape Town, South Africa

Funding

This research was conducted as part of a PhD undertaken by CK at the University of Oxford. It was supported through a scholarship from the Clarendon Fund and St John's College Kendrew Clarendon Award, in partnership with the Nuffield Department of Clinical Medicine.

Author information

Authors and Affiliations

Authors

Consortia

Contributions

Conceptualisation and methodology: CMK, ME, JMcK, RA and CO Analysis, integration and interpretation: CMK, ME, JMcK, RA, JE, AR, CO and the members of the InCARE Stakeholder Group. Writing – original draft: CMK. Writing – review, editing and approval: ME, JMcK, RA, JE, AR, CO and the members of the InCARE Stakeholder Group. The authors read and approved the final manuscript.

Corresponding author

Correspondence to Claire M. Keene.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

All authors have consented to publication.

Competing interest

There are no relevant financial or non-financial competing interests to report.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: Box 1.

Bright et al’s proposed definition of engagement. Figure 1. Frameworks reviewed. Table 1. Stakeholder list. Box 2. Stakeholder engagement process. Table 2. Individual survey answers. Figure 2. Combined Google Jamboards from both group discussions.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Keene, C.M., Euvrard, J., Amico, K.R. et al. Conceptualising engagement with HIV care for people on treatment: the Indicators of HIV Care and AntiRetroviral Engagement (InCARE) Framework. BMC Health Serv Res 23, 435 (2023). https://doi.org/10.1186/s12913-023-09433-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12913-023-09433-4

Keywords