Questionnaire
We employed a modified 'Therapeutic Interventions and Activities Audit Questionnaire' from the YCPM [5] to assess the engagement and competency level. This had the advantage of enabling wide distribution to staff.
The questionnaire was designed by Coventry and Warwickshire Partnership NHS Trust based upon the Care Packages and Pathways Model devised by South West Yorkshire Mental Health Trust. The model has needs-based care clusters related to service users' symptoms, symptom severity, complexity of disorder and chronicity of disorder [5].
It contained 220 items, each one being a specific care activity as identified in the YCP model. The items were broadly categorised into the 10 major care clusters; (1) Assessment activities, (2) Monitoring activities, (3) Enabling activities, (4) Therapeutic interventions, (5) Role support activities, (6) Family/carer interventions, (7) Accommodation, (8) Care co-ordination, (9) Social participation, and (10) Personal and professional care management capabilities. Each care cluster was further subcategorised within the questionnaire to form 47 distinct areas of care; therefore the overarching assessment activities care cluster was further broken down into assessment of mental state, assessment of role functioning, assessment of risk and so on.
Frequency of engagement for each individual care activity was determined by asking participants to state whether they were 'regularly and actively engaged in' specified activities. If they stated that they were, then they were asked to rate their competence level for this activity. The questionnaire adopted Benner's From Novice to Expert scale [6]; participants rated their competence level on 1 - 5 likert scales ranging from (1) novice, (2) advanced beginner, (3) competent, (4) proficient and (5) expert.
To examine determinants of engagement and competency level, we included participants' demography and other variables in the questionnaire, such as time of obtaining the first degree. Two focus groups were conducted with a small sample of Trust staff after the data gathered from the questionnaires had been initially analysed, in order to triangulate findings and enrich interpretation (Mental Health Clinical Care Pathways Project - A final report to West Midlands Strategic Health Authority 2010).
Participants
We selected the Coventry and Warwickshire NHS Trust for our study field as this trust had adopted the use of the Yorkshire Care Pathway in the previous year. We targeted all community-based adult mental health teams (524 staff), including three older adult teams, one specialist older adult and three specialist adult teams.
In September to December 2009 we sent the questionnaire with a cover letter to 387 staff to complete and post back to the research team. One hundred and fifty three staff members returned their completed questionnaires, with a response rate of 39.5%. The range of occupational groups among the participants included: qualified nurses (57.5%), occupational therapists (9.9%), psychologists (7.2%), medical staff (2.6%), counsellor/psychotherapist/therapist (3.9%), social worker (5.2%), and non-qualified support staff (13.2%) (including support, time and recovery workers (STR), community supporters and nurses).
Statistical analysis
We analysed data of engagement and competence levels in total, and in 10 therapeutic activity clusters respectively. We also examined data of 47 subheading clusters where needed. We calculated a total score of engagement activities based on the participant's actual engagement, and accordingly computed the score for each cluster through the total number of activities within each cluster engaged in by the participant.
We added the competence level (score) from each activity in which the participant engaged in total and in their 10 therapeutic activity clusters, and calculated an average score for the total activity and 10 clusters (i.e., the added score was divided by the number of activities engaged).
We examined the distribution of engagement and competence levels among occupational groups and other demographic factors using Chi-square or non-parametric K-W tests for categorical or continuous variables. A Spearman correlation was used to investigate a relationship between the number of engagement activities and competence levels. We employed a logistic regression model to investigate associations of low engagement or low competence (cut-off point at their median level) with determinants factors.
All analyses were performed using the SPSS statistical package (Windows version 16.0; SPSS Inc., Chicago, Illinois).