Measuring performance in allied health professional role substitution models of care: a clinician survey

Background Professional role substitution models of care have emerged as a key strategy to address increasing healthcare demand. Gaining insights from those actively engaged in the process of these models’ implementation and evaluation is pivotal to ensuring sustainability and further successful implementation. The purpose of this study was to describe allied-health clinicians’ perceptions, practice, and experiences of healthcare performance evaluation in professional role substitution models of care. Methods Data were collected via an online platform between 22 June − 22 July 2022 using a combination of convenience and network-based sampling of allied-health clinicians involved or interested in the implementation and evaluation of professional role substitution models of care. Clinicians answered 25 questions which consisted of demographic and targeted questions regarding performance evaluation across six domains of healthcare quality (effectiveness, safety, appropriateness, access & equity, continuity of care, and cost, efficiency, productivity & sustainability). Results A total of 102 clinicians accessed the survey, with 72 providing complete survey data. Eleven allied-health professions were represented, working across twelve specialities in thirteen hospital and health services. Whilst most allied-health clinicians (93–100%) supported measuring performance in each of the six healthcare quality domains, only 26–58% were measuring these domains in practice. Allied-health leadership support (62.5%), clinician drive (62.5%), consumer engagement (50%) and medical support (46%) were enablers whilst a lack of resources (human, time, financial (47%)), healthcare performance frameworks and/or policies (40%) were identified as barriers. Given the opportunity, clinicians would invest the most financial resources in digital solutions as a core strategy to improve performance evaluation. Conclusions Allied-health professionals expressed strong support for principles of performance evaluation, however in practice, performance evaluation is still in its infancy in professional role substitution models of care. Organisations can implement strategies that maximise the enablers whilst addressing barriers identified to improve performance evaluation in these models of care. Supplementary Information The online version contains supplementary material available at 10.1186/s12913-024-10556-5.


Background
In contemporary healthcare systems professional role substitution has emerged as a pivotal paradigm shift, wherein non-medical healthcare professionals, such as physician assistants, nurses, and allied health practitioners, assume responsibilities traditionally reserved for medical doctors or specialist physicians [1].This has become a key strategy to address the escalating demands on healthcare services and the impending scarcity of medical workforce [2,3].Furthermore, this shift is closely aligned with the principles of value-based healthcare (VBHC), a philosophy centred on the imperative to strengthen healthcare systems for sustainability while simultaneously delivering high-quality care [4][5][6].The essence of quality healthcare lies in its ability to provide safe, timely, cost-effective care that demonstrably yields positive patient outcomes [4,7,8].
Over the past two decades, the deployment of allied health professional role substitution models has surged, enhancing patient access to care across diverse medical specialties and liberating medical specialists to focus on more complex or urgent cases.Furthermore, a growing body of evidence highlights the safety and effectiveness of these allied health models, and they are also well-received by patients [9][10][11].Nonetheless, the literature supporting these models often falls short in terms of comprehensive performance measurement, lacking the robust evaluation required to gauge their impact effectively [9,10].A recent systematic review underscored the inadequacy of performance measurement in professional role substitution models, particularly concerning established constructs and domains of healthcare quality, such as effectiveness, safety, access, appropriateness, continuity, efficiency, sustainability, and cost of care [7].
For allied health professional role substitution models to gain credibility as viable, legitimate, and effective alternatives in healthcare delivery, it is imperative to demonstrate that the experiences of receiving and providing care do not deteriorate when compared to traditional medical models.Performance evaluation and transparent reporting are instrumental in enhancing implementation, delivery, adoption, sustainability and outcomes of these models [4].At present, there is a glaring absence of agreed-upon approaches for measuring or monitoring the performance of allied health professional role substitution models in practice, necessitating a systematic and coordinated approach to ascertain their value in healthcare delivery settings.This endeavour is vital for accumulating the evidence needed to inform continued or future commissioning of these services [5,7,8,12].
Beyond the traditional purview, there exists a compelling case for a broader perspective-one that acknowledges and validates measures of paramount importance to patients, clinicians, and other stakeholders, including healthcare managers, policy makers, and payers.Allied health professional role substitution models often not only meet but surpass patients' expectations by delivering cost-effective care, timely access, effective treatments, reassurance, enhanced care experiences (e.g., time spent with clinicians, clinical expertise, professional demeanour, and interpersonal skills) [11,13].However, limited research has delved into allied health clinicians' perceptions and experiences concerning performance evaluation aimed at demonstrating the quality of care delivered.Gaining insights from those actively engaged in these models' implementation and evaluation processes is pivotal to ensuring further successful implementation and sustainability.
This study seeks to address these critical gaps by (i) elucidating allied health clinicians' perceptions, practices, and experiences related to performance evaluation; (ii) identifying the enablers, and barriers that shape robust performance evaluation; and (iii) pinpointing and prioritising key strategies capable of bolstering performance evaluation within allied health professional role substitution models.

Study design and participants
An anonymous online survey was designed to include a combination of closed and open-ended questions.The target population for this study comprised allied health clinicians who were actively engaged or had an interest in the implementation, evaluation, and performance measurement of professional role substitution within the healthcare context of Queensland, Australia which has approximately 35 000 allied professionals [14].In 2014, Queensland Health initiated an expanded scope of practice strategy, leading to the development of approximately 133 distinct care models as reported in the 2018-2019 annual activity data reports [1,15,16].

Survey development and validation
To bolster the survey's validity and reliability, a validation process was undertaken following the seven-step protocol outlined in the Association of Medical Education in Europe (AMEE) guide "Developing Questionnaires for Educational Research: AMEE Guide No. 87" [17].This process encompassed the following steps:

Survey content
The survey (Additional file 1) included sociodemographic questions and addressed clinicians' perceptions, practices, and experiences of performance evaluation in professional role substitution (including perceived barriers and enablers).Participants were asked to rank five potential strategies known to support performance evaluation, by distributing $100 of hypothetical funding across these.This was modelled on an Australian Bureau of Statistics question used to consider policy preferences.The 25-question survey consisted of multiple-choice, categorical and Likert scale options to reduce participant burden and risk of incomplete answers.Open ended questions were integrated to capture unanticipated responses [19].

Ethical consideration
This study adhered to the principles outlined in the Declaration of Helsinki and received ethical approval from both the Gold Coast Hospital and Health Service (HREC/2020/QGC/62,104) and Griffith University (GU Ref No: 2020/876).The study also followed the Consensus-Based Checklist for Reporting of Survey Studies (CROSS) [20].

Survey administration
Data collection was facilitated through the REDCap online survey management platform.A combination of convenience and network based sampling was adopted, with invitations and survey links distributed via email through various State, hospital, and professional networks, including Allied Health Professions Queensland (AHPOQ) and expanded scope of practice professional working groups.Respondents were also invited to share links within their networks Participants were provided with a study information sheet outlining the study's objectives and procedures, emphasising the voluntary nature of participation.A reminder email was sent three weeks after the initial invitation.Participants were offered the opportunity to enter a draw for one of three AUD$100 gift vouchers.No personally identifiable information was collected and contact details of participants expressing interest in further research or the gift voucher draw were not linked to survey responses.Completion of the survey implied consent.

Study population
Between June 22 and July 22, 2022, a total of 102 individuals accessed the survey, and 72 completed surveys were received (see Fig. 1).Forty-one surveys were completed by allied clinicians that were currently working in professional role substitution models of care and 31 were completed by clinicians in the planning stages of implementation.Due to the self-enrolment approach used in the study, it was not possible to estimate a response rate.There were no significant differences observed in terms of age (p = 0.08), allied health discipline (p = 0.263), years of experience as allied health professionals (p = 0.394), or education level (p = 0.699) between those who completed the survey and those who did not.However, individuals who completed the survey were more likely to be male (p = 0.009) and already involved in a professional role substitution model of care (p = 0.008).

Representation
A broad representation from 13 out of 16 Queensland hospital and health services was achieved, with over twothirds of responses originating from the State's most populous hospital and health services, including Gold Coast (26.3%),Metro North (20.8%), and Metro South (19.4%).
Table 1 provides an overview of the demographic characteristics of the respondents, who tended to be older and possess more professional experience compared to the average allied health workforce in Queensland [14].56% ordering tests, 22% reporting test results, 78% interpreting investigation results, 71% providing provisional diagnoses, 95% delivering patient management and treatment, 90% offering patient education, 22% prescribing or ordering medications, 73% making referrals to other health professionals, 78% discharging patients to GPs, and 20% performing procedures.

Professional role substitution model of care details
For those allied health clinicians yet to establish a professional role substitution model of care, there was notable interest in podiatry high-risk foot prescribing,

Allied-health clinicians' perceptions on measuring various domains of healthcare quality
Survey respondents were presented with six domains of healthcare quality (effectiveness, safety, appropriateness, access & equity, continuity of care, and cost, efficiency, productivity & sustainability) drawn from the National Academy of Medicine (NAM), formerly recognised as Institute of Medicine (IOM) healthcare quality dimensions [5], the Australian health performance framework [8], Levesque and Sutherland's integrated performance evaluation framework [12], and a systematic review on measuring performance in professional role substitution models of care [7].Most respondents believed that measuring six domains of healthcare quality was either "very important" or "important".Safety (94.4%) and effectiveness (86.1%) were perceived as highly important, whereas 52.7% considered continuity and integration of care to be very important (Fig. 2).

Measuring performance in practice
While respondents expressed support for the principles of performance evaluation across healthcare quality domains, this support did not consistently translate into practice.58%of all respondents that completed the survey reported measuring the effectiveness of their MoCs.A smaller proportion (26%) measured continuity and integration of care.Allied-health clinicians actively working in role substitution MoCs were more likely to measure effectiveness (90.6% vs. 61.9%,p = 0.01) and cost (75.0%vs. 23.8%,p < 0.001) in their current roles compared to those who had not yet implemented a professional role substitution service.Only a third of those measuring performance reported using a framework, primarily relying on locally developed key performance indicators.Figure 3 provides an overview of the domains that were measured in practice, and Additional file 2 offers examples of measures used by clinicians.

Enablers and barriers to measuring performance in professional role substitution MoC
Enablers and barriers to performance evaluation in healthcare were identified among survey respondents.Figure 4 shows the proportion of respondents in the survey who identified the presented contextual factors as enablers or barriers in their practice.Key enablers included allied-health leadership support, identified by 62.5% of clinicians, clinician drive and motivation to promote performance evaluation (62.5%), consumer engagement (50%), and medical support (46%).Major barriers included a lack of resources (47%) and absence of healthcare performance frameworks and/or policies (40%).Allied-health clinicians involved in professional role substitution models of care were more likely to identify allied-health leadership (p = 0.04) and medical support (p = 0.001) as enablers compared to those yet to establish such models.

What allied-health clinicians would be willing to spend on different strategies to support performance evaluation
Allied-health clinicians allocated significantly more financial resources to digital solutions facilitating systematic and standardised data collection, analysis, and reporting ($26.3,95% CI [21.3, 31.3],p = 0.02) and the development of resource toolkits ($23.7,95% CI [20.6, 27.0], p = 0.02).In contrast, less investment was directed towards financial incentives ($4.81, 95% CI [3.1, 6.6], p < 0.001).The mean allocated funding for the development of frameworks ($21.9, 95% CI [17.9, 25.8], p = 0.33) and for education and training ($23.0,95% CI [18.7-27.5],p = 0.16) did not significantly differ from the expected equal distribution of funding.These results were consistent regardless of whether participants were already working in professional role substitution or planning to establish such models.

Dissemination strategies for performance evaluation findings
Among the allied-health clinicians who measured performance various strategies were employed to report and disseminate findings of performance evaluation (Fig. 5).The most common approach was presenting findings at local professional meetings (e.g., health service physiotherapy department meetings).Notably, allied health clinicians engaged in professional role substitution models of care were more likely to present their findings at specialty meetings (p = 0.02), State-wide forums (p = 0.001), conferences/seminars (p = 0.005), and in peer-reviewed journals (p = 0.002).

Discussion
This study reveals an increase in the adoption of alliedhealth professional role substitution models of care (MoC) over the past decade, with implementation across a wide range of medical specialties.However, despite this growth, the translation of principles supporting healthcare performance evaluation into everyday practice remains limited.Only half of the surveyed clinicians reported measuring one or more domains of health quality.This finding aligns with our recent systematic review, which underscores the substantial gaps in performance evaluation of non-medical professional role substitution MoCs [7].Although there is near-universal consensus on the potential benefits of performance evaluation in healthcare, there is limited evidence to guide this practice, and issues concerning strategic alignment and criteria for selecting measures continue to be subjects of debate [21].Inadequate performance evaluation may impact adoption and scalability of models of care.Where models of care are not effective there is a risk of continued provision of suboptimal quality care with potential impact on patient outcomes.
Previously identified challenges that can stifle implementation of performance evaluation warrant consideration.Time, manpower and cost associated with the process of undertaking performance evaluation are often reported as significant barriers, being limited within the constraints of healthcare systems [22][23][24].Our study echoes this, with clinicians indicating a lack of resources as an ongoing barrier to performance evaluation.However, allied-health leadership, medical leadership and support, and clinician drive were identified as important enablers.This likely reflects the broad stakeholder engagement associated with the implementation of the allied health expanded scope of practice strategy in Queensland Health [1,15,16].
There is growing advocacy for active patient involvement in the care delivery process, as this has been associated with improved healthcare quality, outcomes, and cost containment [25].Importantly, our study underscores the often-overlooked importance of consumer engagement in performance evaluation.This highlights the need to move beyond mere rhetoric concerning patient engagement during the planning and implementation of initiatives.Methods to encourage and monitor patient engagement when measuring healthcare performance should also be considered.Overall strategies may include patient education, shared decision making, encouraging patients to take initiative to learn and take ownership of their care and encouraging patients to actively engage in healthcare improvement strategies, of which performance evaluation is a key component [25].
Clinicians indicated that satisfying regulatory requirements or providing financial rewards for performance were less important aspects of measuring performance.Consistent with this, clinicians allocated the least financial resources to incentivising strategies to improve performance evaluation.There is evidence to suggest that financial rewards have mixed effects on performance, with some studies indicating that recognition is a more important reward for healthcare professionals [26].This suggests that healthcare leaders and policy makers need to focus on other strategies to encourage performance evaluation in the models.
Notably, investment in digital platforms emerged as the preferred clinician-identified strategy to support and promote performance evaluation in professional role substitution MoC.Given the opportunity, clinicians would allocate more financial resources to this.Information systems have the potential to provide essential data that can be used in healthcare performance assessment and management.However, assessment is likely to be opportunistic, reflecting availability of data rather than alignment with organisational strategic goals [27].This makes it difficult to determine the true value of new healthcare delivery models, including allied-health professional role substitution MoC.Technology can be used to automate and streamline data collection with potential to provide real-time feedback.This can lead to a more efficient and transparent performance evaluation process that benefits patients, clinicians and the healthcare system [27].
While there is consensus on the need to measure, analyse, and disseminate healthcare performance findings, there is little agreement on how data should be reported and whether different formats are required for different audiences [27].Healthcare is rich in evidence-based innovations, including allied-health professional role substitution MoCs [28].However, even when such innovations are successfully implemented at one site, their uptake at other locations is often slow, if it occurs at all [7].This study found that evaluations of allied-health professional role substitution MoCs were primarily disseminated at a local level.Innovative MoCs that do not effectively report on performance may have limited long-term impact.Localised reporting, rather than wider publication, may be overlooked in literature reviews and evidence syntheses often used to inform policy and practice [29].There maybe benefit integrating research frameworks when implementing new MoC, which clinicians in this study also highlighted as an enabler to performance evaluation.This study highlighted very limited use of external audits (review that is conducted by a party not associated with the company or department) to evaluate performance of professional role substitution MoC.This is an area that warrants further attention with potential to improve performance evaluation, reduce bias, as well as build on the quality and validity of evaluation efforts.

Strengths and limitations
This study is the first to our knowledge to explore alliedhealth clinicians' practice, perceptions, and experiences with measuring performance of professional role substitution MoC in any location.We have identified barriers and enablers to inform strategies to help clinicians in these roles improve performance evaluation.This is integral to in-service transformation and ensures the highest quality of patient care through transparency, accountability, and credibility [30].Whilst it was not possible to determine the response rate, there was a high level of engagement of the study population with 72 respondents across approximately 133 distinct models reported in the State.A further strength is that the responding cohort were derived from a range of hospital and health services across a large state in Australia, from both tertiary and regional/rural settings, and representing clinicians from a range of disciplines and medical specialities.
As with most surveys, respondents with biases may have self-selected into the sample.The survey population was restricted to clinicians working on or intending to implement professional role substitution MoCs within an Australian state with an established expanded scope of practice strategy.Thus, the generalisability of survey results to other states or countries should be considered when interpreting the results.Future studies could extend survey to a wider range of settings.In measuring performance in healthcare, the perspectives of various stakeholders are valuable.Thus, future research should explore the perspectives of other stakeholders, including managers, health executives, medical practitioners, and policymakers.

Conclusions
In conclusion, allied-health professionals working in or interested in professional role substitution MoCs express support for principles of performance evaluation.However, this support does not consistently translate into practice, with inadequate performance evaluation observed across the various domains which reflect healthcare quality.Organisational strategies can maximise enablers while addressing the identified barriers to improve performance evaluation in these models of care.

Fig. 2
Fig. 2 Allied-health clinicians' perception of importance of measuring various domains of healthcare quality (n = 72)

Fig. 4 Fig. 3
Fig. 4 Proportion of allied-health clinicians reporting on various barriers and enablers to measuring performance in practice.(n = 72)

Fig. 5
Fig. 5 Dissemination strategies used by allied-health clinicians to report performance evaluation (n = 42)

Table 1
Participant characteristics