Table 5 Qualitative feedback for improving the audit process

1. Address issues relating to diversity of consumer representation

• The diversity of workshop members’ experiences (see description in Stage 3) was not formally described to workshop attendees

“When you think about it … we're pretty savvy consumers… I did wonder about, how our [PEMAT] responses might actually stand up in the real world… what actually is meaningful for [people from other communities or with other health conditions] may be completely different for me…” (consumer involved in PEMAT assessments and workshops)

• Many of the consumers perceived a need for a more diverse range of consumers in the project. For experienced consumers it could sometimes be challenging to speak on behalf of other consumers

2. Provide greater opportunity for unstructured feedback and set clear expectations about the consumers’ role in the project

• Some participants wanted to give more detailed input on individual resources that went beyond the scope of the assessment tools and time constraints of the workshop discussions

“I found the [PEMAT] tool restrictive in some ways because it couldn't actually grade the document on its on its real value because you are sort of constrained by the requirements of the tool… How is this document going to affect me? What am I going to learn from this that I don’t already know?” (consumer involved in PEMAT assessments and workshops)

• This may reflect that perceptions that project tasks were unexpected. Consumers discussed that when they helped develop health information, they typically went through resources one-by-one

“I sit on [another] health literacy committee and… we just go through like 5 documents… we go through from top to bottom…not rushing through just because we want to check a box. That we did it, but it's actually holistic.” (consumer involved in workshops only)

• In preparation for the workshops, all workshop attendees were provided with background material explaining the project’s scope and aims. However, consumer feedback suggested that this material was not well understood, or may not have been accessed

“…it might have been better to actually have had a lot more time and a bit more background on how what this actually was going to be presented… and to be more informed on the day when the group session actually happened.” (consumer involved in workshops only)

3. Adapt the health literacy assessment tool to suit consumer needs

• Consumers who used the PEMAT (Stage 2) saw value in having a systematic tool to assess health information

“I can understand that we do need to have it in some systematic way, but it can't be totally an academic system” (consumers involved in PEMAT assessments and workshops)

• Resolving discussions with another consumer was considered a very useful and valuable part of the PEMAT assessment

“…the buddy system was really useful.” (consumer involved in PEMAT assessments and workshops)

• They also discussed that the PEMAT could be more consumer-friendly, and quicker to use. This could include reducing the number of PEMAT items or adjusting item phrasing

4. Simplify presentations that summarise audit data and methods

• Consumers and staff attending the first workshop felt that the PEMAT and SHeLL Editor results gave an interesting perspective

“I was quite surprised to see just how many of the resources didn't meet the criteria and had too high reading grades.” (Consumer involved in workshops only)

• Consumers and some staff felt too much information was presented during the first workshop. This reduced allocated time for discussion

“I think the presentations were very clear and … I took away that people had a good handle on what was being done and what they needed to do as part of the workshops.” (Staff involved in workshops)

• Some attendees at the first workshop felt that this information overload may have made it harder to achieve the aims and take ownership of the project

“I think the first workshop … was very long and quite intense… A lot of listening [about]… context that needed to be shared, but that just made that workshop an awful long time.” (Staff involved in workshops)

• Participants were more positive about the second workshop and preferred the opportunity to discuss individual resources. This activity aligned closely with the expectations described in theme 2