Measuring organisational readiness for patient engagement (MORE): an international online Delphi consensus study

Background Widespread implementation of patient engagement by organisations and clinical teams is not a reality yet. The aim of this study is to develop a measure of organisational readiness for patient engagement designed to monitor and facilitate a healthcare organisation’s willingness and ability to effectively implement patient engagement in healthcare. Methods The development of the MORE (Measuring Organisational Readiness for patient Engagement) scale was guided by Weiner’s theory of organisational readiness for change. Weiner postulates that an organisation’s readiness is determined by both the willingness and ability to implement the change (i.e. in this context: patient engagement). A first version of the scale was developed based on a literature search and evaluation of pre-existing tools. We invited multi-disciplinary stakeholders to participate in a two-round online Delphi survey. Respondents were asked to rate the importance of each proposed item, and to comment on the proposed domains and items. Second round participants received feedback from the first round and were asked to re-rate the importance of the revised, new and unchanged items, and to provide comments. Results The first version of the scale contained 51 items divided into three domains: (1) Respondents’ characteristics; (2) the organisation’s willingness to implement patient engagement; and (3) the organisation’s ability to implement patient engagement. 131 respondents from 16 countries (health care managers, policy makers, clinicians, patients and patient representatives, researchers, and other stakeholders) completed the first survey, and 72 of them also completed the second survey. During the Delphi process, 34 items were reworded, 8 new items were added, 5 items were removed, and 18 were combined. The scale’s instructions were revised. The final version of MORE totalled 38 items; 5 on stakeholders, 13 on an organisation’s willingness to implement, and 20 on an organisation’s ability to implement patient engagement in healthcare. Conclusions The Delphi technique was successfully used to refine the scale’s instructions, domains and items, using input from a broad range of international stakeholders, hoping that MORE can be applied in a variety of healthcare contexts worldwide. Further assessment is needed to determine the psychometric properties of the scale.


Background
Promoting patient engagement in routine clinical care is considered one of the pillars of modern medicine [1,2]. In the UK, patient engagement has received renewed policy support in the form of the National Commissioning framework [3], and the Commissioning for Quality and Innovation (CQUIN) framework, where both enable commissioners to reward implementation initiatives. Further, several national programmes have emerged to stimulate shared decision-making (SDM) and selfmanagement, which are important aspects of patient engagement [2]. This includes the National Voices and MAGIC websites, which provide a broad evidence base on how to best support self-management and SDM [4], and projects exploring and promoting the implementation of SDM [5,6]. Still, despite increasing policy commitments, routine adoption remains difficult [7,8]. Various definitions of patient engagement have been introduced [9]. Angela Coulter, in her book 'Engaging patients in healthcare' [2], defines patient engagement as "working together to promote and support active patient and public involvement in health and healthcare and to strengthen their influence on healthcare decisions, at both the individual and collective level". Sometimes, the term 'patient engagement' is used interchangeably with terms such as 'patient involvement' or 'patient activation'. However, these terms do not reflect all aspects of patient engagement, including the reciprocity between patients and health professionals or healthcare organisations [2,10].
Patient engagement in healthcare at the individual level can occur in direct care interactions, for example through SDM and at the collective level in the form of organisational design and governance [10]. Most work to date has focused on engaging patients at the individual level, e.g. barriers to implementing SDM at the health professional's and patient's levels have been explored [11,12]. When promoting engagement in organisational activities such as the commissioning, planning, design, delivery, evaluation, and reconfiguration of health services, patients and members of the public will need to be enabled to be engaged [13]. So far, studies have focused on barriers to engagement among health professionals and patients, but the impact of organisational barriers and facilitators on the spread and adoption of patient engagement in healthcare has not been closely investigated. A complex whole system change such as patient engagement in healthcare would require its adoption by the organisation as a whole [14]. It is widely recognised that organisational changes are challenging, and often unsuccessful [15]. Some have suggested that half of all failures to implement organisational change can be ascribed to insufficient organisational readiness for change [16]. Organisational readiness for change can be defined as a state of being willing and able to take action [17]. A timely assessment of organisational readiness can facilitate effective implementation by enabling a tailored implementation strategy. Despite extensive searches, to the best of our knowledge, no tools to assess organisational readiness for patient engagement in healthcare exist. A systematic review of the literature on the conceptualisation and measurement of organisational readiness for change by Weiner and colleagues yielded 43 instruments [18]. Many of these scales were not specific to the healthcare context, and data on reliability or validity was often lacking. Recently, Weiner and colleagues published a measure of organisational readiness for change in healthcare settings. However the items in this generic scale were not specifically focussed on achieving patient engagement [19].
Given this gap, we initiated work to develop such a measure. This study addressed the following research question: 'Which domains and items should be included in a scale to monitor and facilitate a healthcare organisation's willingness and ability to effectively implement patient engagement in healthcare?'. As we were striving to develop a scale that would fit in a variety of healthcare contexts, the development of this scale included the consultation of a broad range of international stakeholders in patient engagement.

Study design
A Delphi study was performed to seek consensus on the nature, number, and wording of the items to include in the scale. The Delphi approach consists of an iterative process that aims to combine the perspectives of a panel of experts into a group consensus [20]. It was decided to conduct two Delphi rounds, in anticipation of competing priorities and time constraints of relevant stakeholders in the field of patient engagement [21]. In two rounds of online questionnaires, a panel of stakeholders was consulted to provide feedback about the evolving set of items. The anonymous responses from participants in round 1 were fed back to them in round 2.

Participants
To maximise the generalisability of the results to a variety of healthcare contexts worldwide, we planned to include healthcare managers, policy makers, medical clinicians, nurse clinicians, other health professionals, administrative staff, patients and patient representatives, and researchers from various countries. Members of the research team nominated multi-disciplinary stakeholders on the basis of their interest and expertise in patient engagement. However, these experts all work in the field of patient engagement and most were acquainted with the researchers. In an effort to include a wider range of perspectives, additional open invitations were circulated through national and international mailing lists including the Evidence Based Medicine mailing list, the Shared Decision Making Network, the Shared Decision Making listserv, and the NHS CHAIN network. The invitations provided a brief outline of the study, an enclosed participant information sheet, and a link to the online survey. Consent was inferred by participants' completion of the survey. Ethical approval was granted on 3 September 2013, by the University of Hertfordshire's Ethics Committee for Studies Involving the Use of Human Participants (LMS/SF/UH/00024).

Theoretical framework
A literature search on appropriate theories to underpin the development of the proposed new scale identified Weiner's theory of organisational readiness for change in healthcare settings [17]. This theory was developed based on a comprehensive review of the literature on how organisational readiness has been defined and measured [18]. Weiner postulates that organisational readiness for change can be viewed as a psychological state that is shared by the members of the organisation, and in which they feel committed to implementing an organisational change (change commitment or 'willingness to change'), and are confident in their collective abilities to do so (change efficacy or 'ability to change'). Further, Weiner defined three key determinants of an organisation's ability to change: task demands ('do we know what it will take to implement this change effectively?'), resource availability ('do we have the resources to implement this change effectively'), and situational factors ('can we implement this change effectively given the situation we currently face?').

Questionnaire development
A literature search was conducted to identify any publications describing the development or validation of relevant scales. Four instruments that were broadly related to our planned approach were identified. This included a hospital self-assessment inventory for patientand family-centred care [22], an organisational selfassessment tool for patient-centred care [23], a checklist for organisational readiness for improving patient care experience [24], and an organisational communication climate assessment toolkit [25].
We designed the domains of the 'Measuring Organisational Readiness for patient Engagement' (MORE) scale according to Weiner's theory and developed items that were aligned with the theory's key constructs, and in part derived from existing scales [22][23][24][25]. This was an iterative process involving regular discussions in the research team.

Round 1 survey
The round one survey included a brief introduction to the aim of the study and a summary of the development process and theoretical underpinnings. Participants were asked to rate each of the proposed items on a four-point Likert scale. For the items about the stakeholders targeted by the scale, participants were asked to rate whether each of the proposed stakeholders should be involved (1 = avoid; 4 = definitely involve). For all other items, participants were asked to rate the importance of the items (1 = not important; 4 = very important). Furthermore, for each subset of items and at the end of the questionnaire, participants were given the opportunity to provide rewording suggestions, additional comments, or questions. Email addresses were collected for inclusion in round 2. Participants were also asked to complete basic demographic questions. The survey was open for three weeks and two email reminders were sent to the stakeholders on the contact list.

Round 2 survey
Round 1 participants were invited to complete a second survey, in which feedback was provided about the items' ratings of importance (percentage of participants who thought an item was important or very important) and about the changes made based on the qualitative feedback. Participants were invited to re-rate the importance of the items and to provide additional rewording suggestions, comments or questions. The same demographic items were included. The survey was open for three weeks and two email reminders were sent.

Analysis
Following round 1, the research team discussed the qualitative feedback received, including rewording suggestions per item, suggestions to add new items and more general comments or questions. Items were revised if at least two participants suggested it, or if the researchers agreed that the item would benefit from rewording or merging. For example, for item a in domain 1 'Informing patients about their condition or potential health issues', two participants suggested changing 'informing' to 'discussing' and one participant suggested using the phrase 'engaging patients in..'. For round 2, the item was reworded accordingly to 'Engaging patients in discussing their condition or potential health issues'.
Following the two survey rounds, a decision on whether to retain items in the scale was made based on the ratings of importance in round 2. The ratings were summarised using percentages and the views of all participants were given equal weight. If at least 70% of participants rated the importance of the item in the lower two categories (not important or somewhat important), or in the higher two categories (important or very important), it was determined that consensus was achieved and the item would be removed or retained, respectively. A 70% threshold was considered appropriate, considering that the MORE scale will undergo further testing, and is consistent with related research using a Delphi approach [26,27]. If no consensus was achieved, the research team decided whether or not to retain an item in the scale, basing this decision on qualitative feedback from the participants where possible. For example, no consensus was achieved on whether to retain gender, age and length of employment in domain 1 of the scale. The research team decided to retain these items to examine the representativeness of the stakeholders completing the tool, supported by qualitative feedback from multiple participants emphasising the importance of capturing as many different perspectives as possible.

Development of the first draft of the scale
A total of 51 items were developed, divided into three domains. The first domain contained items related to the background of the stakeholders involved in completing the scale, to ensure that these stakeholders were representative of the entire workforce (see Table 2). Following Weiner's theory, the second domain was designed to assess the organisation's willingness to implement patient engagement and included 14 items reflecting the different aspects of patient engagement (at the individual and collective levels) (e.g. 'Informing patients about their condition or potential health issues'). These items were partly derived from existing scales [22][23][24][25] and complemented with newly developed items to reflect all aspects of patient engagement. The third domain was designed to assess the organisation's ability to implement patient engagement, in terms of tasks (9 items, e.g. 'Developing a shared organisational vision for patient engagement'), resources (10 items; e.g. 'Expertise in patient engagement'), and situational factors (7 items; e.g. ' Alignment of patient engagement with organisational priorities'). Again, items in these domains were partly derived from existing scales [22][23][24][25] and complemented with newly developed items. In addition, the online questionnaire contained six questions about the demographics of the Delphi survey participants. It was developed using Bristol Online Survey [28]. To support face and content validity, the survey was pilot tested among colleagues.

Survey results
Participants' ratings of importance are shown in Table 2. In round 1, 222 rewording suggestions, 238 other suggestions and 23 general comments were provided. In round 2, 91 rewording suggestions, 42 other suggestions, and 8 general comments were made. Below, the results of the survey are presented per domain of the scale, with each round described separately.

Domain 1: stakeholders involved in completing the scale Round 1
Between 95 and 99% of all participants agreed that senior and junior managers, clinicians and other health professionals should be involved in completing the scale. The majority of participants also agreed to include the following staff categories: receptionists, other staff in administration, and other staff. Several participants commented that executives or members of the organisation's Board should also be involved in completing the scale. The items 'Senior Managers' and 'Junior Managers' were reworded to 'Executives/Board of Directors' and 'Managers'. Other feedback related to involving patients in completing the scale. However, given many of the items do not apply to individual patients, a recommendation to involve patient representatives will therefore be included in the instructions. In the future, we will consider developing a separate version of the scale specifically intended for patients. Assessing the gender, age, and length of employment of stakeholders completing the scale was believed to be important or very important by 38%, 50%, and 61% of the survey participants. Based on the qualitative feedback, three new items were added in round 2: (1) 'role in the organisation' , (2) 'discipline (e.g. cardiology)' and (3) 'ethnicity'.

Domain 2:
The organisation's willingness to implement patient engagement Round 1 Most participants (83-99%) considered that it would be 'important' or 'very important' to include all 14 items when assessing an organisation's willingness to implement patient engagement. Six items were reworded, which essentially consisted of representing the patient's voice more strongly and using plainer language. Furthermore, the two items about eliciting patients' preferences and taking patient preferences into account were combined and reworded into ' Asking patients about their health-related preferences and acting upon them'. The two items about provision of written information and decision support tools were perceived to be overlapping and merged into 'Supporting patients with additional health information resources (e.g. access to patient groups and decision support resources)'. Two new items were added in order to emphasise the importance of including self-management as a key aspect of patient engagement, as well as the role of all members of the organisation in treating patients as partners. Further, some participants suggested broadening the definition of 'patients' by emphasising the role of families and carers. Others believed that the term 'patient' would not apply to healthy people using health services (e.g. maternity services or screening). An explanation of this term will be provided in the instructions of each domain of the scale.

Round 2
Similarly to round 1, all 14 items were rated very highly (85-100%). For the nine combined and/or reworded items, ratings were similar or higher than round 1 ratings. The newly added items were rated very highly (99%). Based on participants' suggestions, three of the items were reworded to emphasise that communication should be tailored to individual patients. The newly added item about treating patients as partners, with respect and consideration for their individual needs, was removed based on the comment that this would be difficult to operationalise and measure.

Domain 3: The organisation's ability to implement patient engagement Tasks
Round 1 All ten items in this domain were rated very highly (83-99%). Six items were reworded, three of which were clarified by adding examples. Two items about tailoring communication and consultations to individual patients' needs were removed because of perceived duplication.
Round 2 Ratings of importance were consistently high or higher than in round 1 (92-100%). Following participants' comments, the two items about sharing the organisational vision for patient engagement were merged into: 'Sharing the organisational vision for patient engagement with employees, patients, and the public (e.g. information on intranet, information in waiting areas)'.
In line with this change, the item about monitoring patient engagement and giving feedback to employees was reworded accordingly: 'Monitoring patient engagement in the organisation and giving feedback to employees, patients, and the public'. Further, the item about solving problems was broadened to encompass both individual issues and organisational improvements.

Resources
Round 1 Participants gave all ten items a high rating of importance (80-98%). Two items about training were combined into the following: 'Training health professionals in patient engagement (e.g. communication and shared decision-making skills)'. We also combined three items about patient education materials, decision support resources, and trained medical interpreters and care coordinators: 'Resources to provide health-related information and support to patients (e.g. access to interpreters, answering questions, helping patients to make decisions). In addition, we reworded five items and added examples. Three new items about access to patient representatives, having resources to support patients in becoming partners, and tools to evaluate the implementation of patient engagement were added.
Round 2 Increased ratings of importance were seen for all seven combined or reworded items (94-100%). In addition, high ratings were obtained for the three new items (79-94%). The items addressing the impact of time on patient engagement were reworded to account for the fact that not only time, but other resources too, would be required. Some participants felt that the item on adaptable systems and processes was unclear. This item was merged with the item about resources to provide health-related information and support to patients: 'Resources to provide health-related information and support to patients to meet their diverse needs (e.g. access to interpreters, answering questions, scheduling appointments)'. Other participants found the item about access to patient representatives confusing. It was merged with the resources to support patients in becoming partners item into 'Resources to support patients in becoming partners (e.g. access to patient representatives, recruitment of representatives, training, coaching, money to pay patients for participation)'.

Situational factors
Round 1 Ratings of importance ranged from 80 to 97%.
The 'timing of implementation item' was removed because of perceived duplication. We reworded the item about 'positive and consistent communication' into 'frequent and consistent communication' to reflect the respondents' belief that communication is not/should not be overly positive. Two items about employee and patient involvement in planning the implementation were reworded to reflect the necessary involvement of employees and patients in all aspects of patient engagement (planning, implementation, and evaluation).
Round 2 Compared to round 1, participants gave very similar or slightly higher ratings (92 to 97%). The item about organisational priorities was reworded to ' Alignment of organisational priorities with patient engagement' to better reflect that patients' priorities should be the organisation's priorities.

Principal findings
This study describes the first steps in the development of a measure of organisational readiness for patient engagement in healthcare (MORE). Guided by Weiner's theory, a first version of the scale was developed by the research team using items derived from related tools. One hundred and thirty one multidisciplinary stakeholders from 16 countries gave their feedback on the proposed instructions, domains, and items in an online Delphi consensus procedure. Based on stakeholders' feedback, the number of proposed items was reduced from 51 to 38 by merging overlapping items and deleting items that would be difficult to operationalise and measure. Participants' feedback was also helpful to word items in plainer language, to clarify items by adding examples, and improve the scale's instructions.

Strengths and limitations
This study is the first to report the development of a theory-based measure of organisational readiness for patient engagement. This work builds on the expertise and experience of the authors in developing measurement scales in the area of patient engagement in healthcare [29][30][31][32], and was complemented by that of the 131 international stakeholders. It is worth noting that 16 countries were represented, therefore promoting the potential applicability of the scale in various healthcare contexts.
Although the broad selection of stakeholders is a strength, it may be considered a weakness due to the inability to establish the exact number and categories of stakeholders approached through open invitations. In addition, while our sampling approach targeted a variety of healthcare contexts, we cannot be sure that all healthcare contexts (e.g. primary care, nursing homes, etc.) were represented in our survey. Further, despite including the CHAIN network, no administrative staff completed the Delphi survey. This is considered a weakness as receptionists and other administrators of healthcare organisations have an important role to play in fostering patient engagement and are often the patient's first port of call. Some groups, especially policy makers, seem to be underrepresented in the survey. While this could potentially be addressed by assigning more weight to the answers of this group, it would be difficult to decide on the weight that should be used, and there are no indications that policy makers rated the importance of the items differently compared to other participants. Another group that may have been underrepresented is patients. Only one of the participants identified himself as a patient. However, 22 participants identified themselves as patient representatives, describing a variety of roles including family members and patient advocates. Finally, the consistently high ratings of importance across all three domains suggest a ceiling effect. To improve the design of future Delphi studies, it has been suggested that ceiling effects can be reduced by replacing the 4-point scale with a 5-point scale, using three positive choices (moderately important, important, and very important) and two negative choices (not at all important, somewhat important) [33].

Results in context
A sizeable proportion of commonly reported barriers to patient engagement in healthcare are admittedly influenced, and occasionally fostered, by organisational factors: Time, continuity of care, workflow, characteristics of the healthcare settings (e.g. noisy environments, lack of privacy), lack of skills to apply SDM and access to training, clinical coordination and inter-professional collaboration, overall attitudes towards patient engagement, competing priorities and targets (e.g. criteria for referral, reimbursement in favour of surgery) [34,35]. If patient engagement is to become a reality in routine care, it is critical to give organisations the means to identify and address those barriers through usable routine measurements. As far as can be determined, MORE is the first measure to target organisational readiness for patient engagement in healthcare. However, other scales have focussed on related constructs. The Organisational Readiness for Implementing Change measure (ORIC) was based on Weiner's theory of organisational readiness for change [19]. While ORIC's initial psychometric assessment, combining a qualitative approach, lab studies and field testing show promises, convergent, predictive and discriminant validity still remain to be tested. Further, one may question the value of lab studies that did not include the target population with limited data in naturalistic settings. By contrast, Wynia's validation study of an organisational communication climate assessment toolkit was conducted among its target users, across 13 geographically and ethnically diverse health care organisations [25]. The analysis suggested that domains and items were reliable and accurately predicted patients' perceptions of the quality of organisational communication.

Future research
Building on the above validation methodologies, and given our intention to develop a measure that can be applied in routine settings as well as being used for research purposes, we intend to prioritise naturalistic testing over simulated validation. We will use a combination of qualitative and quantitative methods. First, the items and domains of MORE will undergo further refinement using cognitive debriefing interviews with a wide range of international stakeholders. MORE was initially conceptualised as a measure that could be used across countries and healthcare settings (with minor adaptation and translation where necessary). We therefore intend to interview stakeholders from the US, UK and from at least one European country. In parallel, observation and field notes will be used to determine how the scale can fit in healthcare settings. Our ultimate goal is for MORE to become a routine measure, one that is not used solely for research purposes but to foster change and improvement and enable organisations to improve their practices, track progress and flag-up potential barriers to patient engagement. Second, we will assess the psychometric properties of the scale in a field study across geographically diverse healthcare organisations. We will assess intra-rater reliability, as well as convergent, discriminant and predictive validity. In accordance with Weiner's theory, we will analyse within group agreement in order to determine whether 'organisational readiness for patient engagement' is indeed a psychological state shared by the members of an organisation.