Development of the Organisational Health Literacy Responsiveness (Org-HLR) self-assessment tool and process
BMC Health Services Research volume 18, Article number: 694 (2018)
The World Health Organization describes health literacy as a critical determinant of health and driver of citizen empowerment and health equity. Several studies have shown that health literacy is associated with a range of socioeconomic factors including educational attainment, financial position and ethnicity. The complexity of the health system influences how well a person is able to engage with information and services. Health organisations can empower the populations they serve and address inequity by ensuring they are health literacy responsive. The aim of this study was to develop the Organisational Health Literacy Responsiveness self-assessment tool (Org-HLR Tool), and an assessment process to support organisations with application of the tool.
A co-design workshop with health and social service professionals was undertaken to inform the structure of the tool and assessment process. Participants critiqued existing self-assessment tools and discussed the likely utility of the data they generate. A review of widely used organisational performance assessment tools informed the structure and self-assessment process. The Organisational Health Literacy Responsiveness (Org-HLR) Framework (with seven domains/24 sub-domains) provided the structure for the assessment dimensions of the tool. The performance indicators were drawn from raw data collected during development of the Org-HLR Framework.
Twenty-two professionals participated in the workshop. Based on the feedback provided and a review of existing tools, a multi-stage, group-based assessment process for implementing the Org-HLR Tool was developed. The assessment process was divided into three parts; i) reflection; ii) self-rating; and iii) priority setting, each supported by a corresponding tool. The self-rating tool, consistent with the Org-HLR Framework, was divided into: External policy and funding environment; Leadership and culture; Systems, processes and policies; Access to services and programs; Community engagement and partnerships; Communication practices and standards; Workforce. Each of these had 1 to 5 sub-dimensions (24 in total), and 135 performance indicators.
The Org-HLR Tool and assessment process were developed to address a gap in available tools to support organisations to assess their health literacy responsiveness, and prioritise and plan their quality improvement activities. The tool is currently in the field for further utility and acceptability testing.
Health literacy has been defined as “the cognitive and social skills which determine the motivation and ability of individuals to gain access to, understand and use information in ways which promote and maintain good health” . Population studies conducted in the United States (U.S.), Canada, Europe and Australia suggest that limited functional health literacy is a public health challenge in these countries [2,3,4,5], and that people with low functional health literacy may have less knowledge about their health conditions and treatments, poorer overall health status, and higher rates of hospitalisation than the rest of the population [6,7,8]. Low functional health literacy may also impact an individual’s ability to participate in decision-making, follow care recommendations, implement health-promoting behaviours, and engage with preventative health services [2, 9, 10].
Studies have also shown a relationship between health literacy and a number of socioeconomic factors, including educational attainment, financial position and ethnicity [5, 11]. A recent Australian study reported that health literacy varies across population groups. In particular, people born overseas, people who speak a language other than English at home, people with limited formal education, and people with chronic conditions were more likely to have low health literacy . However, it must be acknowledged that health literacy is content and context specific. That is, a person’s ability to access, use and understand information is greatly influenced by the context in which they are required to apply the information to make health-related decisions .
The World Health Organization recently acknowledged health literacy as a critical determinant of health and a key driver of citizen empowerment and health equity in the Shanghai Declaration, which establishes a mandate for developing, implementing and monitoring strategies for strengthening health literacy in all populations . Such strategies must extend beyond a focus on the health literacy of individuals, to include strategies that reduce the demands health systems place on people, and minimise barriers to them accessing services and information. By ensuring they are health literacy responsive, health and social care organisations can empower the populations they serve.
Health literacy responsiveness has been defined as “the provision of services, programs and information in ways that promote equitable access and engagement, that meet the diverse health literacy needs and preferences of individuals, families and communities, and that support people to participate in decisions regarding their health and social wellbeing” . Some authors have suggested that health literacy responsive organisations foster a culture that promotes equity and inclusiveness, demonstrate effective leadership and management, and ensure robust data collection, monitoring and communications systems and processes are in place. They also foster effective communication practices, and have a strong commitment to building the capability of their workforce, engaging meaningfully with the communities they serve, and partnering effectively with other health and social service organisations [15,16,17,18].
Health care organisations and health systems must continuously improve to ensure they are effective, efficient and responsive to the needs of populations . Organisational self-assessments can be useful for improving organisational performance and effectiveness by supporting benchmarking and monitoring, guiding continuous quality improvement activities, encouraging collaboration and inclusive problem solving, and promoting self-reflection and organisational learning . They are increasingly being utilised to guide organisational ‘diagnosis’ and needs identification, and facilitate goal setting and quality improvement planning [21,22,23]. Such tools generally consist of a set of performance categories, accompanied by a set of ‘best practice’ statements and examples that set a benchmark for performance assessment and planning .
While the need for organisations to better respond to the health literacy needs of populations has been acknowledged [15, 25,26,27], there are few self-assessment tools available to support organisations to assess their health literacy responsiveness. To date, most of the tools that have been developed for this purpose have been based on the ‘ten attributes of a health literate organisation’ proposed by the U.S. Institute of Medicine . The ten attributes were derived through expert opinion and a synthesis of the literature on health literacy research and practice . The ten attributes are framed as a set of goals, supported by a list of strategies or actions organisations can implement to improve their responsiveness. Four of the attributes relate to communication, in addition to one each on leadership, planning and evaluation, preparing the workforce, involving consumers, meeting the needs of populations, and ensuring easy access. These attributes have been incorporated into several organisational assessment or audit tools outside of the U.S. context, including the Enliven Organisational Health Literacy Self-Assessment Resource , the Health Literacy Review: A Guide , and the Vienna Health Literate Organizations Instrument (V-HLO-I) .
Self-assessment tools are more likely to be effective when they are appropriate in conceptual scope and assessment content, provide diagnostic guidance, and have high validity . Organisational models and frameworks serve as the reference standard for self-assessment . Thus, to inform the self-assessment tool developed in this study, we developed the Organisational Health Literacy Responsiveness (Org-HLR) Framework . In summary, the Org-HLR Tool was developed using data generated from a series of concept mapping consultations with more than 200 professionals working in health and social services sectors. Concept mapping is a mixed methods process that incorporates nominal group techniques, unstructured sorting, multivariate statistical methods and in depth user input [32,33,34].
Concept mapping has been widely used for over 25 years to aid the development of theory, plan for programs and social interventions, evaluate social programs, and develop measures and scales  including conceptual frameworks that have subsequently been found to be robust across settings [36,37,38,39]. It provides an inclusive and participatory approach and can be used to explore complex, multidimensional concepts and is therefore highly suited to assist with generation of the components of a health literacy responsive organisation. We further examined the concept mapping data using hierarchical cluster analysis to derive the domains and sub-domains of the Org-HLR Framework shown in Fig. 1, as well as the performance indicators that make up the self-assessment tool .
The aim of the current study was to develop the Organisational Health Literacy Self-Assessment Tool (Org-HLR Tool), and an assessment process to support organisations with the application of the tool.
This study was undertaken in two parts. The first involved determining the structure of the self-assessment tool and an approach for undertaking the assessment process. The second involved selecting the assessment dimensions and performance indicators for the self-assessment tool. The activities and steps involved in this study are shown in Fig. 2. This study was approved by the Deakin University Human Research Ethics Committee (Study ID: 2012-295).
Co-design workshop to inform the structure of the tool and assessment process
To inform the structure of the tool, and the approach for undertaking the assessment process, we conducted a co-design workshop with health and social service sector professionals. The aim of the workshop was to obtain information on the potentially useful characteristics of a self-assessment tool and assessment process from the perspective of practitioners and managers working within relevant organisational settings and contexts. An invitation to participate was distributed to the member organisations of four Primary Care Partnerships in a metropolitan region of Melbourne, Australia. Primary Care Partnerships are voluntary alliances of health and human service organisations, which work together to improve access to, and coordination of services . A total of 22 professionals participated in the workshop who represented the full range of services in Victoria. They included representatives from hospitals (n = 5), community health services (n = 7), women’s health services (n = 2), local governments (n = 2), Primary Care Partnerships (n = 2), Primary Health Networks (n = 1) and other non-government organisations (n = 3). Participants were made up of people providing direct care to consumers, people managing staff who provide care, and people involved in delivering community based projects. They ranged from health promotion officers to executive managers of organisations. All participants provided written consent.
To commence the workshop, participants were introduced to the Org-HLR Framework, including a brief explanation of the domains and sub-domains. Presentation of the framework assisted to orient the group to the nature of the assessment being discussed. A group activity was then undertaken to obtain perspectives on the characteristics of a useful self-assessment tool and assessment process within their settings. This discussion focused particularly on potential ways a tool might be structured and administered rather than the content of the assessment itself. Participants were provided with four existing self-assessment tools and asked to critique them according to a set of guiding questions, which sought their views on the strengths and limitations of the tools, the potential barriers and enablers to implementing them, the types of data they generate and how useful that would be, and suggestions on ways the tools could be improved. The four tools used in this activity were: i) Assessing Chronic Illness Care (ACIC) Survey: V3.5 ; ii) Enliven Organisational Health Literacy Self-Assessment Resource ; iii) Health Literacy Review: A Guide ; and the Agency for Healthcare Research and Quality (AHRQ) Primary Care Health Literacy Assessment . These tools were selected for their specific focus on health literacy or a related health system issue, and because they varied in their design, structure and mode of administration. The characteristics of these tools are described in Additional file 1. In a second activity, participants were asked to describe the characteristics of an ‘ideal’ self-assessment tool, based on their experience of undertaking assessments and planning improvement activities within their contexts. This second activity allowed the workshop participants to reflect upon the discussion of tool strengths and limitations and highlight to the research team those characteristics they considered particularly important.
Determining the tool structure and assessment process
The feedback provided during the consultation workshop was analysed, using qualitative thematic analysis, to identify the perceived strengths and limitations of the example tools, the perceived utility of the data generated for supporting priority setting and action planning, and the perceived overall utility of the tools for assessing organisational performance in relation to health system issues. These key themes informed decisions regarding the structure of the Org-HLR Tool and assessment process.
We also reviewed a range of widely used organisational performance and self-assessment tools to identify their common characteristics, and to make judgements about their likely strengths and limitations, degree of implementation difficulty, and their potential to produce data that will support quality improvement planning, monitoring and evaluation. This included a review of factors such as clarity of the assessment dimensions, the length of the tools, the mode of administration, the rating systems and data collection methods used, and the types of instructions and guidelines provided to support implementation. We included two widely used organisational performance tools in the review, the Baldrige Criteria for Performance Excellence  and European Foundation for Quality Management (EFQM) Model , as well as more localised self-assessment tools relating to cultural competence [45,46,47], gender equity,  consumer-centred/coordinated care, [41, 49] and other tools. The useful elements of these tools were identified, adapted and incorporated into the structure of the Org-HLR Tool and assessment process.
Selecting assessment dimensions, sub-dimensions and performance indicators for the tool
The seven domains of Org-HLR Framework (derived through prior development work described in Fig. 1) served as the broad assessment categories of organisational performance, while the sub-domains informed the sub-dimensions for each assessment dimension. The performance indicators were selected based on the statements generated through the concept mapping process utilised in the development of the Org-HLR Framework .
The statements generated through the concept mapping process were evaluated to determine a representative set of performance indicators (items) for each sub-dimension. In selecting each item within a dimension, consideration was given to ensuring good coverage, whether a wide range of professionals would be able to interpret and understand the item, whether they would be able to make a judgement about how well their organisation is addressing the item, and whether the item could realistically be operationalised into a concrete action to improve organisational practice and performance. In order to describe the breadth of the concept as expressed by participants during the concept mapping consultations, it was important to retain as many distinctions as possible whilst at the same time minimising the potential response burden associated with having too many assessment criteria. Duplicate statements were removed, conceptually similar statements combined, and a minimal set of candidate items for the assessment tool were selected through consensus by the research team.
Step 4: Review of candidate dimensions, sub-dimensions and performance indicators for the tool
Independent researchers reviewed a draft version of the assessment tool comprising a proposed set of dimensions, sub-dimensions and performance indicators. Their task was to assess whether: i) the titles for each assessment dimension and sub-dimension provided appropriate coverage of the assessment topic; ii) titles appeared easy to interpret by a broad audience; and iii) performance indicators contained within each sub-dimension were both easy to interpret and represented the intended focus of the sub-dimension.
Key themes derived from co-design workshop on the structure of the tool and assessment process
Participants in the co-design workshop reported the following to be important characteristics of a useful self-assessment tool: i) it has relevance and usefulness across a broad range of settings (i.e. community health, hospitals, primary care, local government; ii) it is capable of examining whole of organisation performance; iii) it is administered in a way that supports cross-team and cross-system conversations and action; iv) it generates quantitative and qualitative information that allows for identification of strengths and weaknesses; v) it is accompanied by instructions and guidelines as well as supporting resources and templates; vi) it is logically structured and divided into modules which are well defined and explained; and vii) it can be undertaken in a timely manner, without the support of external facilitators.
Development of the assessment process
Based on the key themes derived from the co-design workshop and a review of existing self-assessment tools and processes, a multi-stage, group based assessment process for implementing the Org-HLR Tool was determined to be the most appropriate. This decision was based particularly on the view expressed by participants that the assessment process should facilitate conversations within the organisation and provide deep insights into the areas of strength and opportunities for development. Participants identified that the process of generating the assessment results was as important as the results themselves, in supporting change to occur within their organisations. Consequently, the assessment process was designed to be completed by organisations through a series of facilitated, multidisciplinary workshops. The proposed process was structured according to the Org-HLR Tool; i) reflection; ii) self-rating; and iii) priority setting.
Part 1: Reflection activity
The reflection activity, conducted over approximately 60–90 min, encourages reflection and discussion about health literacy concepts, the specific health literacy needs of clients and communities, and the organisation’s role in responding to them. It provides an opportunity to orientate members of the organisation to the assessment process and prepare them for the subsequent self-rating and priority setting tasks.
Part 2: Self-rating activity
The self-rating activity, conducted over approximately three to 4 h, enables organisations to assess their health literacy responsiveness against a set of performance criteria, as well as identify specific strengths and weaknesses in organisational capability and performance.
Part 3: Priority setting activity
The priority setting activity, conducted over approximately two to 3 h, supports organisations to prioritise actions and improvement activities based on the areas of weakness identified in the self-rating activity.
Development of the structure and content of the self-assessment tool
The proposed self-assessment tool was divided into three parts, to correspond with the three assessment activities. This was complemented by a user-guide, which provided definitions, instructions and recommendations for undertaking the assessment process, as well as templates to support organisations to document the results and outputs of their assessment activities (User-Guide provided at Additional file 2).
The reflection tool provided a set of questions for guiding reflection and group discussion about health literacy concepts and the specific health literacy needs of the organisation’s clients and communities. The proposed questions were:
How well do we currently understand the concepts of health literacy and health literacy responsiveness?
How well do we currently understand the relationship between health literacy, health literacy responsiveness and consumer experiences and health outcomes?
How well do we currently understand and promote equity, diversity and consumer-centred care?
To what extent do we understand and acknowledge our role in making it easy for consumers and the broader community to access the information, programs and services we provide?
To what extent do we respond effectively to the needs of community members that experience barriers (for example due to financial circumstances, disability, mobility constraints, culture, language, low literacy, distance) to accessing support?
These questions were presented as a guide and organisations are encouraged to adapt the questions to their context.
The self-rating tool was divided into seven assessment dimensions, each of which is made up of 1 to 5 sub-dimensions (24 in total), and 135 performance indicators, as shown in Table 1.
In order to encourage discussion about organisational strengths and weaknesses, a global scoring system was derived. This scoring, applied at the sub-dimension level was regarded as more useful than at the individual performance indicator level. A truncated version of the self-rating tool is provided at Additional file 3, which shows two exemplar performance indicators for each sub-dimension of the tool. The descriptors for the five rating levels of the self-rating tool are described in Table 2.
A template was provided for recording the identified strengths and weaknesses at the end of each assessment dimension. The data recorded in the template is utilised to populate the priority setting tool template and prepare for/inform discussions during the priority setting stage of the assessment process.
Priority setting tool
The priority-setting tool provided a set of questions to guide a discussion on the organisation’s strengths and weaknesses. The proposed questions were:
What do we currently do well to support the health literacy needs of consumers and the community?
What could we do better to support the health literacy needs of consumers and the community?
What system/process/practice improvements need to occur within the organisation to strengthen our responsiveness to the health literacy needs of consumers and the community?
Do we currently have the available expertise, capacity and system capability to implement the required improvements?
After discussing and rating organisational performance using the questionnaire, Participants are then prompted to generate a list of actions that need to be implemented to improve their performance against the seven assessment dimensions (based on the examples recorded during the self-rating activity). A template was provided for participants to rate the level of importance of the actions they identify and the level of resourcing required to implement them. The criteria for rating the priority level were:
This requires immediate action, as this is very likely to have significant impact on our overall performance, and prevents us from making improvements in many other areas (highest);
This requires action, as it may have a significant impact on our overall performance and prevents us from making improvements in some other areas; and
This requires gradual action, as it is not likely to have a significant impact on our overall performance and does not prevent us from making improvements in other areas (lowest).
The proposed criteria for recording the resourcing required are:
Can be achieved with existing resources;
Requires additional staff resources;
Requires additional financial resources; and
Requires additional staff and financial resources.
We sought to develop a self-assessment tool and process that enables organisations to identify their health literacy responsiveness strengths and limitations, and then prioritise and plan health literacy-related system improvement activities. We derived the Org-HLR Tool and process through co-design processes with a wide range of professionals working in the health and social services sectors, in order to ensure its relevance and utility within these sectors in Australia. Consequently, the Org-HLR Tool is comprised of seven assessment dimensions, 24 sub-dimensions (impact areas) and 135 performance indicators, and supported by a comprehensive assessment process, including reflection, self-rating and priority setting activities.
Ford & Evans  propose five key characteristics that influence the effectiveness and suitability of a self-assessment tool within a given context: i) conceptual domain; ii) concreteness; iii) diagnostic guidance; iv) affiliation (has been developed by a credible and respected institution); and v) validity. The development of the Org-HLR Tool ensured that it has strengths across each of these characteristics. Firstly, the tool was derived from a conceptual framework comprised of the elements of a health literacy responsive organisation, therefore its conceptual scope aligns specifically with the requisite assessment areas and overall objectives of a health literacy responsiveness self-assessment. The tool demonstrates concreteness insofar as it has a specific focus on a priori elements of a health literacy responsive organisation and uses concepts, language and terminology that reflect the context and experiences of the intended users. It is accompanied by instructions and templates that seek to increase the utility of the tool by managers and practitioners without the support of external experts or facilitators. The tool intends to provide sound diagnostic guidance by supporting the identification of organisational strengths and limitations, with an emphasis on “actionable improvement opportunities”. Finally, the Org-HLR Tool was developed using a participatory approach involving a wide range of health and social service organisations with clearly defined co-design methodology. All of the above elements support credibility and face validity with intended users.
The Org-HLR Tool is comprised of assessment dimensions and a comprehensive set of sub-dimensions that provide clear actionable areas for improving organisational performance. The empirical development of the assessment dimensions in collaboration with a wide range of professionals is likely to ensure it has validity and utility across a range of health and social service contexts, including community health, women’s health, hospitals, primary care services, local governments and other peak bodies and not-for-profit organisations.
The Org-HLR Tool differs from other health literacy self-assessment tools, in that it is incorporated into a comprehensive assessment process. We expect the proposed assessment process will maximise the utility and effectiveness of the Org-HLR Tool in a number of ways. Firstly, completion of the self-assessment process using group based workshops will enable participation by a diverse range of staff, thereby supporting cross-organisational learning, collaborative ‘diagnosis’ and inclusive decision-making . This is likely to increase staff engagement in the improvement activities that flow from the self-assessment process. The use of qualitative and quantitative data collection methods will enable organisational benchmarking, and monitoring and evaluation of improvements over time [22, 51]. The priority-setting tool will encourage organisations to move beyond the problem identification stage, by incorporating the outputs of the assessment process into organisational planning and implementation activities . Whilst a potential strength of the tool, its length and the requirement that multiple actors within an organisation participate in the process may limit its utility in some settings. Organisations considering undertaking the self-assessment may also find it necessary to orient key staff to the concepts of health literacy and health responsiveness in order to effectively engage them in the assessment process.
Although the Org-HLR Tool was developed for use by organisations in the Australian context, it is likely to have utility in healthcare settings in other countries. Health systems internationally share common features and are increasingly focused on ensuring effective engagement of the communities they delivery care to. It will however be necessary to adapt the tool for use in other contexts where barriers to access take different forms. The Org-HLR Tool and assessment process will need to be tested and validated in Australia, as in other countries to determine the applicability and comprehensibility of the assessment dimensions, and the feasibility of the assessment process. This is particularly important because each healthcare context may not only have a different mix of health services with different professional staff, but terminology to describe elements within services are likely to differ substantially. Healthcare systems that are fragmented, under-resourced or have complex insurance and gatekeeper structures, provide additional challenges to community members seeking care. The role of the Org-HLR Tool in these settings is yet to be tested and may require considerable modification.
We developed the Org-HLR Tool and assessment process to address a gap in the availability and suitability of tools to support organisations to assess their health literacy responsiveness strengths and limitations, and prioritise and plan their quality improvement activities. The tool is likely to be relevant for a broad range of organisations across the health and social service sectors.
The tool is currently in the field for further testing of its utility and acceptability in a range of health and social service organisations across Victoria, Australia. This will allow us to identify the improvements and adaptations required to enhance its utility across these settings.
Assessing Chronic Illness Care
Agency for Healthcare Research and Quality
European Foundation for Quality Management
Organisational Health Literacy Responsiveness
Vienna Health Literate Organizations Instrument
Nutbeam D. Health promotion glossary, World Health Organization; (1998). http://www.who.int/healthpromotion/about/HPG/en/. Accessed 15 Aug 2014.
Institute of Medicine. Health Literacy: A Prescription to End Confusion, Institute of Medicine; (2004). http://www.nationalacademies.org/hmd/Reports/2004/Health-Literacy-A-Prescription-to-End-Confusion.aspx. Accessed 23 May 2013.
Murray S, Rudd R, Kirsch I, Yamamoto K, Grenier S. Health Literacy in Canada - Initial Results from the International Adult Literacy and Skills Survey, Canadian Council on Learning; (2007). https://books.google.com.au/books/about/Health_Literacy_in_Canada_Initial_Result.html?id=1n-PoAEACAAJ&redir_esc=y. Accessed 23 Apr 2013.
Australian Bureau of Statistics. Australian Social Trends: Health Literacy Australian Bureau of Statistics; (2009). http://www.ausstats.abs.gov.au/ausstats/subscriber.nsf/LookupAttach/4102.0Publication30.06.093/$File/41020_Healthliteracy.pdf. Accessed 10 May 2013.
HLS-EU Consortium. Comparative Report on Health Literacy in Eight EU Member States: The European Health Literacy Survey HLS-EU; (2012). http://ec.europa.eu/chafea/documents/news/Comparative_report_on_health_literacy_in_eight_EU_member_states.pdf. Accessed 13 May 2013.
Herndon JB, Chaney M, Carden D. Health literacy and emergency department out-comes: a systematic review. Ann Emerg Med. 2011;57(4):334–45.
Wolf MS, Gazmararian JA, Baker DW. Health literacy and functional health status among older adults. Arch Intern Med. 2005;165(17):1946–52.
Tokuda Y, Doba N, Butler JP, Paasche-Orlow MK. Health literacy and physical and psychological wellbeing in Japanese adults. Patient Educ Couns. 2009;75(3):411–7.
Ishikawa H, Yano E. Patient health literacy and participation in the health-care process. Health Expect. 2008;11(2):113–22.
van der Heide I, Uiters E, Rademakers J, Struijs JN, Schuit AJ, Baan CA. Associations among health literacy, diabetes knowledge, and self-management behavior in adults with diabetes: results of a dutch cross-sectional study. J Health Commun. 2014;19(Suppl 2):115–31.
Paasche-Orlow M, Wolf M. The causal pathways linking health literacy to health outcomes. Am J Health Behav. 2007;31(1):S19–26.
Beauchamp A, Buchbinder R, Dodson S, Batterham RW, Elsworth GR, McPhee C, et al. Distribution of health literacy strengths and weaknesses across socio-demographic groups: a cross-sectional survey using the Health Literacy Questionnaire (HLQ). BMC Public Health. 2015;15:678. Available from: http://www.ncbi.nlm.nih.gov/pubmed/26194350. Cited 3 June 2015
World Health Organization, editor. Shanghai Declaration on Promoting Health in the 2030 Agenda for Sustainable Development. 9th Global Conference on Health Promotion. Shanghai; Geneva: World Health Organization; 2016.
Trezona A, Dodson S, Osborne RH. Development of the Organisational Health Literacy Responsiveness (Org-HLR) Framework in collaboration with health and social services professionals. BMC Health Serv Res. 2017;17(1):513.
Brach C, Keller D, Hernandez LM, Baur C, Parker R, Dreyer B, et al. Ten attributes of health literate health care Organisations: discussion paper. Washington: Institute of Medicine; 2012. Accessed 27 Oct 2013
Parker RM, Hernandez LM. What makes an organization health literate? J Health Commun. 2012;17(5):624–7.
Kripalani S, Wallston K, Cavanaugh K, Osborn CY, Mulvaney S, McDougald S, et al. Measures to assess a health-literate organization. Washington: Vanderbilt Center for Effective Health Communication; 2014.
Dodson S, Good S, Osborne RH. Health literacy toolkit for low and middle income countries: a series of information sheets to empower communities to strengthen health systems. New Delhi: World Health Organization; 2015.
Wolf JA. Organization development in healthcare : conversations on research and strategies. Bingley: Emerald Group Publishing; 2011.
Ritchie L, Dale BG. Self-assessment using the business excellence model: a study of practice and process. Int J Prod Econ. 2000;66(3):241–54.
Miller R, Bedney B, Guenther-Grey C. Assessing organizational capacity to deliver HIV prevention services collaboratively: Tales from the field. Health Educ Behav. 2003;30(5):582–600.
Ford MW. Baldrige assessment and organizational learning: the need for change management. Qual Manag J. 2001;8(3):9–25.
Fountain M. The target assessment model as an international standard for self-assessment. Total Qual Manag. 1998;9(4&5):S95–S9.
Trenerry B, Paradies Y. Organizational assessment: an overlooked approach to managing diversity and addressing racism in the workplace. J Divers Manage. 2012;7(1):11–26.
Koh HK, Baur C, Brach C, Harris LM, Rowden JN. Toward a systems approach to health literacy research. J Health Commun. 2013;18(1):1–5.
DeWalt D, McNeill J. Integrating Health Literacy with Health Care Performance Measurement. Washington: Institute of Medicine; 2013.
World Health Organization Regional Office for Europe. Health Literacy: The Solid Facts. Copenhagen: World Health Organization; 2013.
Thomacos N, Zazryn, T. Enliven Organisational Health Literacy Self-assessment Resource. Melbourne: Enliven & School of Primary Health Care, Monash University; 2013.
Ministry of Health. Health literacy review: a guide. Wellington: Ministry of Health; 2015.
Dietscher C, Pelikan J. Health-literate hospitals and healthcare organizations - results from an Austrian Feasability study on the self-assessment of organizational health literacy in hospitals. In: Schaeffer D, Pelikan J, editors. Health literacy. Bern: Hogrefe; 2017.
Ford MW, Evans JR. Models for Organizational Self-Assessment. Business Horizons. 2002. Available from: https://www.researchgate.net/publication/4884983_Models_for_Organizational_Self-Assessment?enrichId=rgreq-1f7a6e2f3443860e63c540825c5a4e74-XXX&enrichSource=Y292ZXJQYWdlOzQ4ODQ5ODM7QVM6MTg5MTg1MzE1MDU3NjY3QDE0MjIxMTY3MDI2MjQ%3D&el=1_x_2. Accessed 2 Jan 2017.
Trochim WM. An introduction to concept mapping for planning and evaluation. Eval Program Plann. 1989;12(1):1–16.
Trochim WM, Milstein B, Wood BJ, Jackson S, Pressler V. Setting objectives for community and systems change: an application of concept mapping for planning a statewide health improvement initiative. Health Promot Pract. 2004;5(1):8–19. discussion 0
Risisky D, Hogan VK, Kane M, Burt B, Dove C, Payton M. Concept mapping as a tool to engage a community in health disparity identification. Ethn Dis. 2008;18:77–83.
Rosas SR, Ridings JW. The use of concept mapping in measurement development and evaluation: application and future directions. Eval Program Plann. 2017;60:265–76.
Norgaard O, Furstrand D, Klokker L, Karnoe A, Batterham R, Kayser L, et al. The e-health literacy framework: a conceptual framework for characterizing e-health users and their interaction with e- health systems. Knowl Manage E-Learn. 2015;7(4):522–40.
van Randeraad-van der Zee CH, Beurskens A, Swinkels R, Pool J, Batterham RW, Osborne RH, et al. The burden of neck pain: its meaning for persons with neck pain and healthcare providers, explored by concept mapping. Qual Life Res. 2016;25:1219–25.
Vaughn LM, Jacquez F, McLinden D. The use of concept mapping to identify community-driven intervention strategies for physical and mental health. Health Promot Pract. 2013;14(5):675–85.
Osborne RH, Batterham RW, Elsworth GR, Hawkins M, Buchbinder R. The grounded psychometric development and initial validation of the Health Literacy Questionnaire (HLQ). BMC Public Health. 2013;13:658. Available from: http://www.ncbi.nlm.nih.gov/pubmed/23855504
Department of Health and Human Services, Primary Care Partnerships, Department of Health and Human Services (2015). https://www2.health.vic.gov.au/primary-and-community-health/primary-care/primary-care-partnerships. Accessed 1 Apr 2017.
MacColl Institute for Healthcare Innovation. Assessment of Chronic Illness Care, Version 3.5. 2000.
Agency for Healthcare Research and Quality. Primary Care Health Literacy Assessment. Rockville: Agency for Healthcare Research Quality, 2010.
Baldrige Performance Excellence Program. The metrology of organizational performance: how Baldrige standards have become the common language for organizational excellence around the world. United States: Baldrige Performance Excellence Program; 2015.
Nabitz U, Klazinga N, Walburg J. The EFQM excellence model: European and Dutch experiences with the EFQM approach in health care. Int J Qual Health Care. 2000;12(3):191–201.
Andrulis D, Delbanco T, Avakian L, Shaw-Taylor Y. Conducting a cultural competence self-assessment. New York. https://www.samhsa.gov/sites/default/files/programs_campaigns/samhsa_hrsa/cultural-competence-self-assessment.pdf.
Ministry for Children and Families. Cultural Competence Assessment Tool. Vancouver: Ministry for Children and Families; 2009. http://www.multiculturalmentalhealth.ca/wp-content/uploads/2013/10/Enhancing_Cultural_Competency_Resource_Kit11.pdf.
Department of the Premier and Cabinet. A Cultural Inclusion Framework for South Australia. Department of the Premier and Cabinet. Adelaide: Government of South Australia; 2006. https://www.dpc.sa.gov.au/what-we-do/services-for-business-and-the-community/Aboriginal-community-advice-and-support/publications/cultural-inclusion-framework. Accessed 10 Dec 2015
Harvey J, Kindervatter S, Woods A. The gender audit handbook: a tool for organizational self-assessment and transformation. Washington: InterAction; 2010.
Department of Health. ASM prepare - express: active service model practice review planning and reflection tool. Melbourne: Victorian Government; 2010.
Anderson DL. Organization development: the process of leading organizational change. 2nd ed. California: Sage Publications; 2012.
Blazey ML. Insights into organisational self-assessments. Qual Prog. 1998;31(10):47–52.
The authors wish to thank the organisations and participants who contributed to this study.
This project was part of a larger project, Ophelia (OPtimising HEalth LIteracy and Access) Victoria, funded by a peer reviewed nationally competitive Australian Research Council (ARC) Linkage Project grant, with partnership funding from the Victorian Department of Health, Deakin University and Monash University. Anita Trezona’s PhD Scholarship was funded through the Ophelia Project. Richard Osborne was funded through a National Health and Medical Research Council (NHMRC) Senior Research Fellowship #APP1059122. The funding body had no influence on the design of the study and collection, analysis, and interpretation of data and in writing the manuscript.
Availability of data and materials
The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.
Ethics approval and consent to participate
This study was approved by the Deakin University Human Research Ethics Committee (Study ID: 2012-295). All participants provided informed written consent to participate in this study.
Consent for publication
All participants provided informed written consent to publish the findings of this study.
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Trezona, A., Dodson, S. & Osborne, R.H. Development of the Organisational Health Literacy Responsiveness (Org-HLR) self-assessment tool and process. BMC Health Serv Res 18, 694 (2018). https://doi.org/10.1186/s12913-018-3499-6