A survey of engagement and competence levels in interventions and activities in a community mental health workforce in England

Abstract

Background

National Health Service (NHS) mental health workforce configuration is at the heart of successful delivery, and providers are advised to produce professional development strategies. Recent policy changes in England have sharpened the focus on competency based role development. We determined levels of intervention activities, engagement and competence and their influencing factors in a community-setting mental health workforce.

Methods

Using a modified questionnaire based on the Yorkshire Care Pathways Model we investigated 153 mental health staff working in Coventry and Warwickshire NHS Trust. A median score of competence was computed across 10 cluster activities. Low engagement and competence levels were examined in a logistic regression model.

Results

In 220 activities, Monitoring risk was the highest rate of engagement (97.6%) and Group psychological therapy/Art/Drama therapy was the lowest engagement (3.6%). The median competence level based on all activities was 3.95 (proficient). There were significant differences in the competence level among professional groups; non-qualified support group (3.00 for competent), Counsellor/Psychologist/Therapist (3.38), Occupational therapists (3.76), Nurses (4.01), Medical staff (4.05), Social workers (4.25) and Psychologists (4.62 for proficient/expert). These levels varied with activity clusters; the lowest level was for Counsellor/Psychologist/Therapist in the accommodation activity (1.44 novice/advance beginner) and the highest for Occupational therapists in personal activity (4.94 expert). In a multivariate analysis, low competence was significantly related to non-qualified community support professions, late time of obtaining first qualification, more frequencies of clinical training, and training of cognitive behavioural therapy. The associations were similar in the analysis for 10 activity clusters respectively.

Conclusions

There was a reasonable competence level in the community-setting mental health workforce, but competence varied with professional groups and cluster activities. New staff and other non-qualified support professions need to receive efficient training, and the training content is more important than frequency to increase level of competence.

Background

The National Service Framework (NSF) for Mental Health in England [1] has recognised that workforce configuration will be at the heart of successful delivery, and providers are advised to produce professional development strategies as the balance of existing skills will need to be adjusted in some areas in order to address a shortage of skills in others. The National Health Service (NHS) Plan [2] strengthened the messages delivered in the NSF, announcing major investment in new models of service: crisis resolution, assertive outreach, and early intervention in psychosis teams, and new roles. Whilst the NSF made specific references to the perceived challenges for professions, the NHS Plan with its new roles and new team models with a specific multidisciplinary focus did not. The message was clear, however, that not only were professions being asked to work in more prescribed multidisciplinary and interdisciplinary models, but also that government policy was looking beyond professional roles to new roles based on competencies.

More recent English Government policy changes have sharpened the focus on competency based role development. In Lord Darzi's final report of the Next Stage Review, High Quality Care for All [3], the government outlined a commitment to develop a common currency system for contracting for mental health services. The Department of Health circular letter [1] announced that this currency system would be available by March 2011, and would be based upon the Care Pathways and Packages methodology known as the Yorkshire Care Pathways Model (YCPM), that had been developed by six mental health trusts in the Yorkshire and Humber and North East regions of England.

The Strategic Health Authority of the West Midlands region of England decided to adopt the YCPM early, utilising the Commissioning for Quality and Innovation Payment Framework [4] to incentivise and encourage provider organisations to allocate all new referrals to the pathway clusters by April 2010. Early development of the currency system was driven utilising tools developed by the productivity programme utilising the YCPM [5]. The differentiation between need and demand in health care commissioning and planning is recognised as challenging. The model attempts to provide a validated clinical decision-making support tool, for the classification of conditions and relevant therapeutic interventions. It is argued that this benefits both service users and health care providers by informing more rational systematic service planning and reducing inter-professional variations. The YCPM allocates service users into care clusters dependent upon their needs or characteristics. By outlining service activities within each cluster, care can be standardised through systematic linking of resources and better identification of service user needs [5]. The YCPM is specifically designed to support multi-professional working; however, there was no clear indication of the competencies required within the workforce to deliver these interventions, or how they may be best organised. In this study, we carried out a questionnaire survey in a community-setting mental health multi-professional workforce in West Midlands region of England, to investigate activities/interventions, engagement and levels of competency and their determinants.

Method

Questionnaire

We employed a modified 'Therapeutic Interventions and Activities Audit Questionnaire' from the YCPM [5] to assess the engagement and competency level. This had the advantage of enabling wide distribution to staff.

The questionnaire was designed by Coventry and Warwickshire Partnership NHS Trust based upon the Care Packages and Pathways Model devised by South West Yorkshire Mental Health Trust. The model has needs-based care clusters related to service users' symptoms, symptom severity, complexity of disorder and chronicity of disorder [5].

It contained 220 items, each one being a specific care activity as identified in the YCP model. The items were broadly categorised into the 10 major care clusters; (1) Assessment activities, (2) Monitoring activities, (3) Enabling activities, (4) Therapeutic interventions, (5) Role support activities, (6) Family/carer interventions, (7) Accommodation, (8) Care co-ordination, (9) Social participation, and (10) Personal and professional care management capabilities. Each care cluster was further subcategorised within the questionnaire to form 47 distinct areas of care; therefore the overarching assessment activities care cluster was further broken down into assessment of mental state, assessment of role functioning, assessment of risk and so on.

Frequency of engagement for each individual care activity was determined by asking participants to state whether they were 'regularly and actively engaged in' specified activities. If they stated that they were, then they were asked to rate their competence level for this activity. The questionnaire adopted Benner's From Novice to Expert scale [6]; participants rated their competence level on 1 - 5 likert scales ranging from (1) novice, (2) advanced beginner, (3) competent, (4) proficient and (5) expert.

To examine determinants of engagement and competency level, we included participants' demography and other variables in the questionnaire, such as time of obtaining the first degree. Two focus groups were conducted with a small sample of Trust staff after the data gathered from the questionnaires had been initially analysed, in order to triangulate findings and enrich interpretation (Mental Health Clinical Care Pathways Project - A final report to West Midlands Strategic Health Authority 2010).

Participants

We selected the Coventry and Warwickshire NHS Trust for our study field as this trust had adopted the use of the Yorkshire Care Pathway in the previous year. We targeted all community-based adult mental health teams (524 staff), including three older adult teams, one specialist older adult and three specialist adult teams.

In September to December 2009 we sent the questionnaire with a cover letter to 387 staff to complete and post back to the research team. One hundred and fifty three staff members returned their completed questionnaires, with a response rate of 39.5%. The range of occupational groups among the participants included: qualified nurses (57.5%), occupational therapists (9.9%), psychologists (7.2%), medical staff (2.6%), counsellor/psychotherapist/therapist (3.9%), social worker (5.2%), and non-qualified support staff (13.2%) (including support, time and recovery workers (STR), community supporters and nurses).

Statistical analysis

We analysed data of engagement and competence levels in total, and in 10 therapeutic activity clusters respectively. We also examined data of 47 subheading clusters where needed. We calculated a total score of engagement activities based on the participant's actual engagement, and accordingly computed the score for each cluster through the total number of activities within each cluster engaged in by the participant.

We added the competence level (score) from each activity in which the participant engaged in total and in their 10 therapeutic activity clusters, and calculated an average score for the total activity and 10 clusters (i.e., the added score was divided by the number of activities engaged).

We examined the distribution of engagement and competence levels among occupational groups and other demographic factors using Chi-square or non-parametric K-W tests for categorical or continuous variables. A Spearman correlation was used to investigate a relationship between the number of engagement activities and competence levels. We employed a logistic regression model to investigate associations of low engagement or low competence (cut-off point at their median level) with determinants factors.

All analyses were performed using the SPSS statistical package (Windows version 16.0; SPSS Inc., Chicago, Illinois).

Results

Among 220 activities, the highest rate of engagement activities among participants was Monitoring risk at 97.6% and the lowest engagement was Group psychological therapy/art/drama therapy at 3.6%. Table 1 shows the 8 highest and 8 lowest engagement activities.

Table 1 Most and least commonly engaged in activities in 153 staff in the adult mental health teams in Coventry and Warwickshire NHS Mental Health Trust

In the analysis of 47 subheading clusters, we found that Monitoring activities (including mental state, risk, physical health and wellbeing), Enabling activities - assertive engagement (including positive engagement to build a relationship of trust and positive engagement to be reliable) and Role support activities, informal counselling (including listening, empathising, reflecting) were the three most common activities with 94.8% of the sampled staff reporting to engage in them. The three least commonly engaged in activities were Role support activities - social support (including accompanying service users to the cinema, sport, art and social clubs) at 12.4%, Assessment activities - diagnosis at 23.5% and Assessment activities - dementia care mapping at 24.2%.

Table 2 illustrates the median competency levels in the total, and in the 10 activity clusters across the occupational groups. The median competence level for all activities was 3.95. It varied with activity clusters; the lowest competence level (3.40) was for therapeutic interventions and the highest (4.08) for care co-ordination activities.

Table 2 Competency levels for total engagement activities and the 10 activity cluster across occupational groups

Among seven professional groups there were significant differences in the total competence level; the lowest to the highest competence levels were for non-qualified support group, Counsellor/Psychologist/Therapist, Occupational therapists, Medical staff, Nurse, Social worker and Psychologist (table 2). The data of the 10 activity clusters showed similar results to those in the total activities, except that psychologists had a lower competency level in Accommodation Support activities (table 2).

The correlation analysis showed no significant association of the averaged competence level with the number of activities engaged in the total (r = -0.03, p = 0.713) and in the 10 clusters excepting a negative relationship in Therapeutic Interventions (r = -0.184, p = 0.028).

Table 3 shows the frequencies of occupational group and other factors and odds ratios (ORs) for the low engagement of total activities. In univariate analysis we found that low engagement was significantly related to non-qualified supporter, counsellor and psychologist. In the multivariate analysis, only the occupational group of psychologist was significantly related to low engagement. The findings from the 10 clusters were similar (table 4). In the activity clusters of Assessment activities, Therapeutic Interventions and Family Support activities, none of those factors was associated with low engagement. In the Monitoring activities, the low engagement was significantly and positively related to non-qualified support workers and psychologists, in the Role Support activities to no clinical training, in the Accommodation activities to psychologist, in the Care Co-ordination to no clinical training, and in the Personal and Professional Care Management Capabilities to psychologists. It was negatively related to Behavioural Family Therapy (BFT) training in the Enabling activities, to social workers in the Role Support Activities, to BFT and protection trainings in the Care Co-ordination activities and to the first degree awarded earlier than 2000 in the Personal and Professional Care Management Capabilities.

Table 3 Distribution of occupation and other factors and odds ratio for low engagement of the all activities

Table 4 Factors in relation to low engagement in the 10 activities†

Table 5 shows the frequencies of occupational group and other factors and ORs for the low competence of total activities. Compared to qualified nurses, non-qualified supporters had a significant low competence, and psychologists had a high competence. The low competence was also significantly increased with later time of obtaining first qualification and CBT training. It was not related to the type of first qualification. In the multivariate analysis, the associations were similar to those in the univariate analysis, except for no significance for psychologist and BFT training (table 5).

Table 5 Distribution of occupation and other factors and odds ratio for low competence of the all activities

The multivariate analysis for the 10 clusters showed similar results to those in the total activities (table 6). Further analysis of the 47 subheading clustering data showed similar results, but less statistical significances (data on request).

Table 6 Factors in relation to low competence in the 10 clusters activities

Discussion

In this NHS community-based mental health workforce study, we found that Monitoring activities, Assessment activities and Role support activities - informal counselling were the most common activities, while Role support activities - social support and therapeutic interventions were less commonly engaged. Although this was across all groups, one would have expected there to be a higher level of engagement in therapeutic interventions. The overall competency level was reasonably good and qualified nurses had an average competency level of 'proficient'. But the competence level varied with professional groups across 10 clusters activities. Low competence level was associated with non-qualified community support professions, late time of obtaining a first qualification, more frequent clinical training, and CBT training.

In an Irish national survey of client care activities carried out by community psychiatric nurses, McCardle et al [7] observed that the common client care activities engaged in by their sample were assessment activities, listening, medication management (including compliance and education), practical support for daily living activities and providing family support while delivery of cognitive behavioural therapies and structured family therapies were limited. These were similar to our findings. Our data expands upon the work of McCardle et al. [7] by examining engagement in therapeutic activities across multidisciplinary teams. Surveys looking at multidisciplinary teams [8] showed that community mental health teams as a whole engaged in assessment activities and supporting carers and family, which were the same as our data, but their study suggested that the most common activity was the provision of therapy or counselling. The variation may be due to differences in mental health workforce staff between the two studies or the fact that the present study looked at the services provided by each mental health team rather than individual staff members' activities.

To our knowledge, our survey is the first study showing competency levels and their determinants for clinical activities in community-setting mental health workforce within the UK. A study conducted by Greaves et al [9] in Australia found self-reported competence levels of occupational therapists working in mental health settings to be comparable to other occupational groups, and significantly higher in community settings, which is consistent with our data in England. Previous studies in general clinical settings [10] have also provided some insight as to the self reported competence levels of non-qualified support staff, and suggested that the changing roles of this occupational group mean that they are increasingly taking on more advanced clinical responsibilities. As a result ongoing supervision of this group is required to maximise the contribution they can make to patient care [11]. As we move into an era of large scale workforce reductions in the NHS, the need to develop cost effective roles to deliver parts of the service will become more evident.

Previous studies [12] have indicated that increased length of professional experience in health care settings was not necessarily indicative of improved performance. But our data, particularly with multivariate adjustment analysis, suggested that those who were more recently qualified may have lower competency levels than those who had a number of years with previous experience in practice. Gauntlett [13] found that a lack of opportunities to practice and training at a level that is too basic, may result in insufficient levels of competence being obtained, particularly for more specialised therapeutic interventions. Our findings that not all clinical training courses, nor a high frequency of training courses undertaken by staff were related to increased competency levels echo Gauntlett's [13] assertion that post-graduate mental health training programmes need greater evaluation to ensure their effectiveness. By aligning service provision along the care pathways, there would be scope to ensure that sufficient ratios of staff are trained to advanced levels on specific therapeutic intervention modalities. Knowing where the specialist skills lie will be key in managing workload when allocating clients to pathways to match supply and demand.

The main contribution of this survey lies in what it tells us about the competency level and its determinants in a representative community mental health workforce in England. The Trust included various professional groups. Its size and advanced state of development in delivering care based on the YCPM meant that it was the most appropriate site for the purposes of the study and that the findings would be sufficiently robust to be indicative of the general state of readiness and education needs of the workforce of large NHS Mental Health Trusts with high community and outreach services, seeking to deliver care using this model. A second strength is that we carried out a standardised YCPM questionnaire and used a multivariate regression model to analyse the data. Our study has limitations. (1) The survey response rate was relatively low, in comparison with other surveys that have been conducted in the NHS setting [14]; this may be attributable to high staff workloads, but the response rate is similar to that in our previous questionnaire study in the community in the UK, where there seemed no significant biases introduced [15, 16]. (2) This was a service evaluation focused on staff perceptions and as such the engagement and competence levels were self-reported by participants in the questionnaire. This may lead to the well-reported social desirability effect [17] and therefore caution should be paid to the reported high level of competence, but it would not necessarily affect findings of low competences in given activities where further training is recommended. The current findings are supported by the focus groups data. Thus the low competence for occupational group and its determinants were ensured. (3) Caution should be exercised in interpreting the causal-result association for the determinants of low competency because the study was cross-sectional. (4) Our studied population is a sample of different professions in the English health care system and caution should be exercised in applying our findings in other mental health systems.

Summary and recommendation, our study has succeeded in profiling the intervention activities engagement and competency levels in the community setting mental health workforce in England. The findings serve as an indicator of the competencies and educational needs of wider Mental Health workforce of the region and nation. However, the findings are based on self-reported levels of competency and given this limitation, we recommend that further studies using this approach would benefit from a more structured use of an expert reference group to evaluate and comment on findings. As identified in No health without mental health [18], services will have £400 million invested over the next four years to make a choice of psychological therapies available for those who need them, expanding provision to most disadvantaged groups. As we move into a new era of Payment by Results (PBR) in mental health, there will inevitably be a sharper focus upon therapeutic interventions and outcomes. This review of current therapeutic activities and interventions will help to ensure that the mental health workforce is sufficiently equipped with the required level and depth of skills. Our findings further indicate a number of ways in which mental health provider organisations may wish to utilise the competency survey, for example where pathways have been aligned to clinical teams to identify areas where skills and competencies may require strengthening. Alternatively the survey could be used to identify where teams could be realigned or reconfigured to provide care along care pathway structures.

The continuing development of the mental health curriculum means that training needs and competency levels will apply to the current workforce, and may have limited generalisability to the workforce being currently trained and qualifying in the near future.

Conclusions

This research has suggested that the self rating competency based questionnaire approach utilised could inform the evaluation and redesign of roles within mental health teams, which is required when implementing a new model of care such as the Yorkshire Care Pathway Model as described here. Within the limits of this study, our findings indicate that mental health staff in this community-setting felt competent to perform most activities itemised in the YCPM. Variations in competence both within and across professional groups in relation to cluster activities can inform the development of bespoke training programmes and offer opportunities to model team competency requirements that are not allied to specific disciplines, supporting the value of multidisciplinary team working. Our survey's results indicate the need for more effective rather than more frequent in-service training to support staff development, in particular more targeted training where there is increasing service demand for example in the provision of certain behavioural therapies.