Skip to main content
Download PDF

Collaborative identification and prioritisation of mental health nursing care process metrics and indicators: a Delphi consensus study

BMC Health Services Research volume 22, Article number: 350 (2022) Cite this article

Abstract

Background

The Irish Office of Nursing & Midwifery Services Director (ONMSD) commissioned the development an updated suite of mental health nursing metrics and indicators for implementation in Irish mental health clinical settings. While measuring care processes does offer the potential to improve care quality, the choice of which mental health nursing metrics to measure presents a significant challenge, both in Ireland and internationally. The provision of safe and high-quality mental health nursing care stems from nurses’ expertise, skills and overall capacity to provide recovery focused care across a range of health care settings. Accordingly, efforts to measure what mental health nurses do depends on the identification of those care processes that contribute to mental health nursing practice. This paper reports on the identification, development and prioritisation of a national suite of Quality Care Metrics (QCM), along with their associated indicators, for mental health nursing care processes in Ireland.

Methods

The study was undertaken over four phases; i) a systematic literature review to identify mental health care process metrics and their associated indicators of measurement; ii) a two-round, online Delphi survey of mental health nurses to develop consensus on the suit of mental health nursing care process metrics; iii) a two-round online Delphi survey of mental health nurses to develop consensus on the indicators to be used to measure the agreed metrics; and iv) a face-to-face consensus meeting with mental health nurses and service user representatives to develop consensus on the final suite of metrics and indicators.

Results

Following these four phases 9 metrics and their 71 associated indicators were agreed for inclusion in the final suite of Mental Health Nursing QCM. These metrics are applicable across the life span and the range of mental health nursing health care settings.

Conclusion

The development of this suite of Mental Health Nursing QCM and their indicators represents an opportunity for the measurement of safe and high-quality mental health nursing care for application in Ireland and internationally. This initial development of metrics and indicators should be followed by a rigorous baseline review of QCM uptake and implementation amongst mental health nurses as part of an ongoing evaluation.

Peer Review reports

Background

The Health Service Executive (HSE) in The Republic of Ireland, along with many other international health care providers recognise the need for high quality, standardised data collection that quantifies nursing and midwifery care processes [, 1, 2]. While such data can contribute to improved understanding of mental health nursing care processes, the choice of which mental health nursing metrics to measure presents a significant challenge in Ireland and internationally [3].

Contemporary mental health care policy emphasises the need for measurement of the contribution of mental health nurses to patient safety and clinical quality improvement [4], 5, 6]. Currently in Ireland and internationally there very limited research in this area and as such no quantifiable means of establishing the contribution of mental health nursing processes to the outcomes of people in receipt of mental health nursing care across the lifespan and healthcare settings. Notable historical efforts to identify and quantify what mental health nurses do include the development of clinical indicators for mental health nursing in Australia and New Zealand in the 2000s. While not developing metrics and their associated indicators, Skews et al. [7] and O’Brien et al. [8] produced clinical indicators and endeavoured to audit these as a means of comparing and benchmarking practice across mental health settings. It is noteworthy that these studies acknowledged the challenge of capturing the totality of mental health nursing practice and of embedding such measurement approaches. In a discussion on mental health nursing culture Slemon et al. [9] note that mental health nursing practice remains focused on mitigating risk and promoting safety, while lacking effective structures. Indeed, even in these priority areas, where mental health nurses utilise rigorous monitoring and recording as means of risk management the evidence is that current approaches lack efficacy [10].

In response to the identified need the Office of Nursing and Midwifery Services Directorate in Ireland commissioned a national research study to establish a consensus on how nursing and midwifery care processes that should be measured (HSE 2018a). This national study aimed to develop care process metrics and indicators that where possible would align with evidenced-based clinical practice guidelines and standards. This study produced a suite of seven QCM reports that outline these metrics and associated indicators in the healthcare areas of Mental Health, Midwifery, Children’s Community/Public Health, Acute, Older and Intellectual Disability (HSE 2018b). This paper reports the development and prioritisation of a national suite of QCM, and their associated indicators, for mental health nursing in Ireland.

Methods

This study comprised four discreet phases outlined below. With all phases overseen by a work stream working group made up of mental health nurses from a range of grades, specialities and geographical areas, service user representation and an academic lead (n = 30). The work-stream working group members were invited to participate in the quality care metrics project by letter informing them of the study and inviting them to nominate registered mental health nurses of all grades from their area to participate in the study. The group was provided with regular updates by email and teleconference and met face to face four times over the course of the project; before the Delphi study commenced, during the Delphi study and after the Delphi study had completed, to agree the final suite of metrics.

Phase 1

A systematic literature review to identify mental health nursing metrics that have been used in this area and the indicators for same. The following inclusion criteria were applied; i) Participants: registered nurses or midwives working in any of the seven identified work-streams, or persons in receipt of nursing or midwifery care from eligible work streams; ii) Exposure: relating to nursing or midwifery quality care processes (metrics or indicators). For the purposes of this study a quality care process metric is defined as a quantifiable measure that captures quality in terms of how (or to what extent) nursing or midwifery care is performed in relation to an agreed standard. A quality care process indicator is defined as a quantifiable measure that captures what nurses or midwives are doing to provide care in relation to a specific tool or method (Foulkes et al. 2011 [11]); iii) Outcomes: measurable quality process or processes in use or proposed for use; iv) Type of study: any.

Phases 2 and 3

A two-round online Delphi survey of mental health nurses to develop consensus on metrics to be prioritised and a two-round online Delphi survey of mental health nurses to develop consensus on indicators for the prioritised metrics. At the end of the first two rounds, the metrics were identified and at the end of Round 3 and 4, the indicators for those metrics were identified. A purposeful convenience sample of mental health nurses working in the mental health services in Ireland were invited to partake in the Delphi survey. Inclusion criteria for the study included any registered mental health nurse working in the HSE mental health services in Ireland, with the ability to give informed consent. Exclusion criteria for the study included any registered mental health nurse not working in the HSE mental health services in Ireland. There were 1102 responses across four Delphi survey rounds.

All mental health nurses who met the inclusion criteria for the study were invited to participate in the study following a comprehensive national advertising campaign via posters, information sheets, word of mouth and presentations at national nursing conferences and events. Participants indicated their interest in the study via email to the research team.

The Delphi data collection and analysis provided group consensus on what metrics (Rounds 1 & 2) and associated indicators (Rounds 3 & 4) should be used. Open fields were provided in Rounds 1 and 3 to allow participants to comment on and suggest additional potential metrics and indicators. Responses to each round were collated then redistributed to participants for further comment in successive rounds. Each round had a response closing date of 21 days after the date of invitation. E-mail reminders were sent to anyone who did not respond by day 7 from the date of invitation. Numbers of participants for each round of the Delphi are presented in Table 1.

Table 1 Nursing and midwifery judgement framework tool
Full size table

Phase 4

Face-to-face consensus meeting with key stakeholders to review the findings and build consensus on the final suite of metrics and indicators. Phase 4: Face-to-face meeting with key stakeholders to build consensus on final metrics and indicators.

This phase comprised of a face-to-face meeting with attending members of the work stream working group (n = 20). The work-stream group members were invited to review and build consensus on the quality care process metrics and indicators developed from the Delphi surveys. Participants were provided with a Nursing and Midwifery Judgement Framework Tool adapted from Flenady et al. [12] to guide their determination on metric/indicator inclusion in the final suite of mental health QCMs (see Table 2).

Table 2 Overall Delphi responses by grade and role
Full size table

Ethics

Ethical approval was granted by the Research Ethics Committee National University of Ireland Galway. All potential participants received a study information sheet, which outlined the purpose of the study, the risks and benefits of participation, confirmed participation was entirely voluntary and the likely time commitment. They were also informed they could ask questions at any point and given contact details to that end. All participants had to explicitly indicate their consent to participate by clicking on the ‘I agree’ button at the end of the online participant information sheet before accessing the survey. For phase 4, consensus meeting participants were given a participant information leaflet. Written consent to participate was then obtained at the meeting.

Results

Systematic review

The literature search was undertaken as a national collaboration across all 7 work streams (Devane et all 2019 [13], Doody et al. 2019 [14], Murphy et al. 2019 [15]). The aim of the review was to identify quality care process metrics and their associated indicators, and to identify the current evidence base.

Eight databases were systematically searched including: Pubmed, Embase, PyscINFO, ASSIA, Cumulative Index to Nursing and Allied Health Literature (CINAHL), Cochrane Database of Systematic Reviews (CDSR), Cochrane Central Register of Controlled Trials (CENTRAL), and Database of Abstract of Reviews of Effects (DARE).

To maintain contemporary relevance the search was undertaken between 1st January 2007 and 1st January 2017, in English language where full text were available. For this purpose a systematic review procedure was adapted using the search terms nurs*:ab,ti OR midwi*:ab,ti AND (‘minimum data set’:ab,ti OR indicator*:ab,ti OR metric*:ab,ti OR ‘quality measure*’:ab,ti) AND [english]/lim AND [2007–2017]/py. The search was not limited to study design but widened to comprise all types of sources including grey literature.

Covidence software (Cochrane 2016) was utilised to manage the retrieved studies. After duplicates were removed, each title was reviewed independently by at least two members of the national academic teams. Disputes were settled by discussion and negotiation. At full text screening, any included studies were tagged to the specific work-streams. Full-text studies relevant to each work-stream were then reviewed by two reviewers (NB and AH for mental health nursing) from the appropriate work-stream. The complete process flow diagram for the systematic literature review is presented in Fig. 1. Search and selection flow diagram.

Fig. 1
figure 1

Search and selection flow diagram

Full size image

Systematic review findings

The search conducted across eight databases resulted in 15,304 citations. Following removal of duplicates, 7,524 unique references were identified. Following title and abstract screening, 218 sources were retained, including those identified from hand searching and the grey literature, for full-text screening. Following full text screening, 112 sources were included upon the basis that they met the study’s inclusion criteria. These sources were reviewed for reference to mental health nursing care processes and quality processes (see Fig. 1). Eight studies were identified as relevant to mental health nursing. Two researchers reviewed these independently for quality and content. Disagreements were resolved between the reviewers and a third reviewer consulted if required. Two from the database search [16, 17] both of which referred to measurable approaches to mental health screening, undertaken by mental health nurses. From the grey literature, the Judgement Support Framework [18] a guidance document designed so support quality in mental health care processes implementation and monitoring.

Following the systematic review process, the Mental Health Work Stream Working Group met to discuss the potential metrics extracted from the systematic literature review. At this stage it was agreed that, the pre-existing suite of mental health nursing care process metrics [19] be included in discussion for possible inclusion. Following this discussion, 16 potential mental health nursing metrics were agreed for inclusion in Round 1 of the Delphi survey: 1. Medication Storage and Custody, 2. Management of Controlled Drugs (MDA Drugs), 3. Medication Administration, 4. Ensuring Correct Prescription of Medication, 5. Assessment and Personal Details, 6.Nursing Care Plan, 7.Nursing and Midwifery Board of Ireland (NMBI) Record Keeping, 8. Provision of Required Information for Service Users & Carers, 9. Discharge Planning, 10. Service User Experience, 11. Screening and Evaluation of Mental Health Needs, 12. Use of psychotherapy/nonpharmacological therapeutics i.e. talk therapies, 13. Use of psychiatric medications side effect profile, 14. Care of the dying, 15. Communication of Healthcare Team, 16. Therapeutic Communication with service users.

Phases 2 and 3 The Delphi consensus process

Delphi-round 1

All those who expressed interest in participating were sent email invitations from SurveyMonkey® to participate. A web link was also created as an additional data collector. Round 1 of the Mental Health Metrics Delphi was launched on the 6th of June, 2017 and remained open for 21 Days. The Delphi survey consisted of consent to participate via an: I agree or do not agree button as well as questions regarding demographics. Participants were asked to rate each of the 16 included metrics from 1–9 (1-Not Important, 9-Very Important). SurveyMonkey® participants were updated weekly according to both Community Health Organisation (CHO) area and nursing grade and circulated to project officers and directors of mental health nursing to track participation rate. Following removal of duplicates and inclusion of only those who provided an email address, there were a total of 290 participants in Round 1.

Delphi round 2

All who expressed interest and whose responses provided data from Round 1 were sent email invitations via SurveyMonkey® to participate in Round 2 of the Mental Health Metrics Round 2 Delphi. Participants were also sent confidential emails prior to the start of Round 2 with PDF copies of their individual Round 1 survey responses to allow them to re-rate the metrics based on both their responses and the group’s responses. Round 2 of the Delphi was launched on the 11th of July 2017 and remained open for 21 Days. Following removal of duplicates and inclusion of only those who provided an email address there were a total of 233 participants. Round 2 participants were asked to rate 20 proposed metrics that had either been maintained from Round 1, added from open fields in Round 1 of re-worded following review by the work stream working group.

Delphi round 3

This round of the Delphi survey was open to new participants that had not previously participated in Round 1 or Round 2. These participants were asked to rate 80 potential indicators that could be used to measure the seven metrics retained from Rounds 1 and 2. Email invitations via SurveyMonkey® were sent to those who had previously completed Round 1 or Round 2 in addition to any new expressions of interest. Round 3 of the Delphi was launched on the 22nd of August and remained open for 21 days. Following removal of duplicates and inclusion of only those who provided an email address there were a total of 233 participants.

Delphi round 4

Email invitations for Round 4 of the Delphi survey were only sent to those who provided data in the Round 3 survey. Participants were also sent confidential emails prior to the start of Round 4 with PDF copies of their individual Round 3 survey responses to allow them to re-rate the indicators based on both their responses and the group’s responses. Round 4 of the Delphi was launched on the 3rd of October 2017 and remained open for 21 Days. Following the removal of duplicates and inclusion of only those who provided an email address there were a total of 143 participants.

Delphi rounds data analysis

As noted participants across the Delphi rounds were asked to indicate the importance of the metrics and indicators using a 9-point Likert scale. At the completion of Delphi round 2 (metrics) and Delphi round 4 (indicators) consensus on inclusion was determined where 70% or more participants rated the metric or indicator as 7 to 9 and less than 15% of participants rated the metric as 1 to 3. These criteria were used consistently across the four rounds to best represent the participants’ judgements over the rounds [20]. The 70% importance rate is also consistent with approaches to developing core outcome sets in routine healthcare http://www.comet-initiative.org/.

Phase 4

Consensus meeting participants voted on each metric and indicator as a yes or no to be included using the Poll Everywhere App. Metrics and indicators were required to receive a vote of 70% or higher to be included for the final suite in mental health quality care metrics. Follow-up discussions and multiple rounds of voting were used where necessary. Following this process, the final agreed suite of nine mental health nursing process metrics and their indicators were finalised (see Table 3):

Table 3 Final agreed metrics (n = 9) and indicators (n = 71) with Delphi round 4 consensus percentages
Full size table

Discussion

The 9 metrics and 71 indicators presented in Table 3, represent the main findings of this study. The metrics and indicators included in the final suite show considerable synergy with literature indicating areas where mental health nurses and service users have identified the need to develop the knowledge, skills and practice of mental health nurses [21]. By identifying prioritised areas where mental health nursing processes can be measured this research contributes to efforts to quantify and improve practices that mental health nurses, service users and carers representatives value. This understanding of these stakeholders’ views on key areas of nursing process indicate potential areas for the development of mental health nursing practice and education.

It should be noted that metrics as applied to health care practices arise from the world of business where metrics were devised as means of setting targets and measuring attainment of these [22]. Pencheon [23] suggests that within healthcare settings, metrics as measured by their indicators illustrate areas of practice performance and the degree to which expectations are being met. Hence, these can be used as a quality control measure in practice. Typically, metrics and their associated indicators are used to show benchmarking or attainment against agreed standards, with a view to ensuring and improving the quality of care [, 24,25,26]. It should be noted that while there are clear potential benefits to the implementation of metrics, there is ongoing concern that care processes, in this case the day to day business of mental health nursing cannot be adequately measured [3]. The concern being that the interpersonal day to day activities delivered by mental health nurses are hard to measure and that mental health nurses do not work alone making their activities as distinct from multi-disciplinary team activities hard to isolate and quantity [27].

In this study, despite the lack of applicable evidence based metrics and indicators identified from the systematic review, the grey literature identified valuable starting points for the deliberations of the work-stream working group, in identifying areas of importance to mental health nursing in the Irish context. The literature sources that were identified ranged from full procedure guidelines with some underpinning evidence through to checklists of areas pertaining to practice to be measured.

Central to the conduct of this research and complimented by the rigorous research process outlined is the level of engagement with service user and family representatives and a broad cross-section of mental health nurses across grades, practice settings and geographical areas in Ireland. This ensured that mental health nursing QCMs and associated indicators arose from genuine consensus. Crucially there was service user and family representation from the outset, across the work stream working group deliberations through to the final Phase 4 consensus meeting, ensuring maximum possible involvement in the ongoing research. This approach reflects the policy emphasis in Ireland that mental health service provision should arise from genuine partnership between service users and service providers [28,29,30].

Despite the lack of applicable evidence identified from the systematic review every effort was made to ensure the quality of the mental health nursing metrics and indicators included in the final suite. The evaluation tool used identified four key attributes of metrics and indicators these being process focused, important, operational and feasible. The rigorous research design employing 4 Phases; systematic literature review, online Delphi survey of mental health nurses and consensus meeting with key stakeholders, means that the finally agreed metrics and indicators can be considered to be process focused with clear applicability mental health service users and mental health nurses in practice. Importantly, as noted in the systematic review section, not all of the agreed metrics and indicators have supporting clinical standards and an evidence base although they do have a strong practice evidence base. This understanding specifically impacts on feasibility, the fourth component of the judgement framework tool. This suggests that there is a need for robust piloting and evaluation of these metrics and indicators before rolling out service wide to identify potential unintended impacts and barriers [31]. Consequently, this initial development of metrics and indicators should be followed by a rigorous baseline review of QCM uptake and implementation amongst mental health nurses as part of an ongoing evaluation.

Conclusion

Achieving consensus on what mental health nurses do and how best to measures those processes in an important element of ongoing efforts to improve the safety and quality of mental health nursing practice. Having achieved its aim in producing a suite of mental health nursing metrics and indicators, there is now the opportunity to incorporate routine measurement into ongoing efforts to improve the quality of mental health nursing care. Any implementation of these metrics and indicators could provide valuable evidence of mental health nurses contribution to safe, quality, care processes.

Availability of data and materials

The datasets used and/or analysed during the current study available from the corresponding author on reasonable request.

References

  1. Kings Fund (2017) Quality improvement in mental health London: The King’s Fund.

  2. Health Service Executive (HSE) (2014). Report of the Quality and Safety Clinical Governance Development Initiative: Sharing our learning. Dr Steevens’ Hospital: Dublin.

  3. Maben J, Morrow E, Ball J, Robert G, Griffiths P. High Quality Care Metrics for Nursing. London: Kings College London; 2012.

    Google Scholar 

  4. Health Service Executive (HSE) Mental Health Division (2017a). Best Practice Guidance for Mental Health Services. Supporting you to meet Regulatory Requirements and towards Continuous Quality Improvement. Dublin: HSE. https://www.hse.ie/eng/services/list/4/mental-health-services/mental-health-guidance/best-practice-guidance/documents/best-practice-guidance-for-mental-health-services.pdf

  5. Health Service Executive (HSE) (2016). Advancing Recovery in Ireland. A guidance paper on implementing organizational and cultural change in mental health services in Ireland. Dublin: HSE. http://hdl.handle.net/10147/613321

  6. Health Service Executive (HSE) (2012). A Vision for Psychiatric/Mental Health Nursing: A shared journey for mental health care in Ireland. Dublin: HSE. https://www.hse.ie/eng/services/publications/nursingmidwifery%20services/a%20vision%20for%20psychiatric%20mental%20health%20nursing.pdf

  7. Skews G, Meehan T, Hunt G, Hoot S, Armitage P. Development and validation of clinical indicators for mental health nursing practice. Aust N Z J Mental Health Nurs. 2000;9:11–8.

    Article  CAS  Google Scholar 

  8. O’Brien A, Boddy J, Derrylea H, O’Brien A. Clinical indicators as measures of mental health nursing standards of practice in New Zealand. International Journal of Mental Health Nursing. 2004;13:78–88.

    Article  PubMed  Google Scholar 

  9. Slemon A, Jenkins E, Bungay V. Safety in psychiatric inpatient care: The impact of risk management culture on mental health nursing practice. Nurs Inq. 2017;24: e12199. https://doi.org/10.1111/nin.12199.

    Article  PubMed Central  Google Scholar 

  10. De Santis M, Myrick H, Lamis DA, Pelic CP, Rhue C, York J. Suicide-specific safety in the inpatient psychiatric unit. Issues Ment Health Nurs. 2015;36:190–9.

    Article  PubMed  Google Scholar 

  11. Foulkes M. Nursing metrics: measuring quality in patient care. Nurs Stand. 2011;25(42):40-5.

  12. Flenady V, Wojcieszek AM, Fjeldheim I, Friberg IK, Nankabirwa V, Jani JV, Myhre S, Middleton P, Crowther C, Ellwood D, Tudehope D. eRegistries: indicators for the WHO Essential Interventions for reproductive, maternal, newborn and child health. BMC Pregnancy Childbirth. 2016;16(1):293. https://doi.org/10.1186/s12884-016-1049-y.Accessed:12.10.2017.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Devane D, Barrett N, Gallen A, et al. Identifying and prioritising midwifery care process metrics and indicators: a Delphi survey and stakeholder consensus process. BMC Pregnancy Childbirth. 2019;19(1):198. Published 2019 Jun 10. https://doi.org/10.1186/s12884-019-2346-z.

  14. Doody O, Murphy F, Lyons R, et al. Development of nursing quality care process metrics and indicators for intellectual disability services: a literature review and modified Delphi consensus study. BMC Health Serv Res. 2019;19:909. https://doi.org/10.1186/s12913-019-4749-y.

  15. Murphy F, Doody O, Lyons R, et al. The development of Nursing Quality Care Process Metrics and Indicators for use in Older Persons Care Settings: A Delphi-Consensus Study Journal of Advanced Nursing. 2019;75(12). https://doi.org/10.1111/jan.14126.

  16. Grabowski DC, Aschbrenner KA, Rome VF, Bartels SJ. Review: Quality of Mental Health Care for Nursing Home Residents: A Literature Review. Med Care Res Rev. 2010;67(6):627–56. https://doi.org/10.1177/1077558710362538Accessed:16.03.2018.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Yackel E, McKennan M, Fox-Deise A. Nurse-Facilitated Depression Screening Program in an Army Primary Care Clinic: An Evidence-Based Project. Nurs Res. 2010;59(1):S58–65. https://doi.org/10.1097/NNR.0b013e3181c3cab6.

    Article  PubMed  Google Scholar 

  18. Mental Health Commission (2016). Judgement Support Framework Version 3. Mental Health Commission: Dublin

  19. Health Service Executive (HSE) (2015). Guiding Framework for the Implementation of Nursing and Midwifery Quality Care-Metrics in the Health Service Executive Ireland. HSE. Dublin.

  20. Hsu C, Sandford B. The delphi technique: making sense of consensus. Pract Assess Res Eval. 2007;2007(12):10.

    Google Scholar 

  21. Bee P, Playle J, Lovell K, Barnes P, Gray R, Keeley P. Service user views and expectations of UK-registered mental health nurses: A systematic review of empirical research. Int J Nurs Stud. 2008;45(3):442–57. https://doi.org/10.1016/j.ijnurstu.2007.02.008.

    Article  PubMed  Google Scholar 

  22. Cusack, E., Dempsey Ryan, D., Kavanagh, C., & Pitman, S. (2014). An Evaluation of the Development and Implementation of a Nursing and Midwifery Metrics System in HSE Dublin North Healthcare Services, available: https://www.hse.ie/eng/about/who/onmsd/nmpdu/nmpddn/an-evaluation-of-implementation-of-nursing-metrics.pdf Accessed: 13.12.2016

  23. Pencheon, D. (2008). The good indicators guide: understanding how to use and choose indicators. NHS Institute for Innovation and Improvement. available: https://www.england.nhs.uk/improvement-hub/wp-content/uploads/sites/44/2017/11/The-Good-Indicators-Guide.pdf Accessed: 16.03.2018

  24. Gage W, Heywood S, Norton C. Measuring quality in nursing and midwifery practice. Nurs Stand. 2012;26(45):35–40. https://doi.org/10.7748/ns2012.07.26.45.35.c9208Accessed:16.03.2018.

    Article  Google Scholar 

  25. Royal College of Nursing (RCN)(2012) Nursing Dashboards- Measuring Quality, Publication no: 004 198, available: https://www.rcn.org.uk/professional-development/publications/pub-004198 Accessed: 16.03.2018

  26. Royal College of Nursing (RCN)(2009) Measuring for quality in health and social care An RCN position statement, Publication no: 003 535, available: https://www.rcn.org.uk/professional-development/publications/pub-003535 Accessed: 19.02.2018

  27. Smith, J.H.F. (2012). Has the alphabet changed? Revised quality assurance guidance for the NHSCSP. Cytopathology, 23, 39. available: http://onlinelibrary.wiley.com/doi/https://doi.org/10.1111/cyt.12011/pdf

  28. Higgins, A. (2008) A Recovery Approach within the Irish Mental Health Services: A Framework for Development. Dublin: Mental Health Commission.

  29. McDaid, S. (2013). Recovery: What you should expect from a good quality mental health service. Dublin: Mental Health Reform.

  30. Watts M, Downes C, Higgins A. Building Capacity in Mental Health Services to Support Recovery. An exploration of stakeholder perspectives pre and post intervention. Dublin: School of Nursing and Midwifery Trinity College Dublin; 2014.

    Google Scholar 

  31. Campbell SM, Kontopantelis E, Hannon K, Burke M, Barber A, Lester H. Framework and indicator testing protocol for developing and piloting quality indicators for the UK quality and outcomes framework. BMC Fam Pract. 2011;12(1):85–95. https://doi.org/10.1186/1471-2296-12-85.

    Article  PubMed  PubMed Central  Google Scholar 

  32. Collins, P., Naughton, L., Heslin, R., & Ryan, M. (2016). Advancing Recovery in Ireland. A Guidance Paper on Implementing Organisational and Cultural Change in Mental Health Services in Ireland. National Office for Advancing Recovery in Ireland. HSE: Mental Health Division. Dublin. http://hdl.handle.net/10147/613321

  33. Health Service Executive (HSE) Mental Health Division (2017b). A National Framework for Recovery in Mental Health Services. A national framework for mental health service providers to support the delivery of a quality, person centred service. Dublin: HSE. https://www.hse.ie/eng/services/list/4/mental-health-services/advancingrecoveryireland/national-framework-for-recovery-in-mental-health/recovery-framework.pdf

  34. McCance T, Telford L, Wilson J, Macleod O, Dowd A. Identifying key performance indicators for nursing and midwifery care using a consensus approach. J Clin Nurs. 2012;21(7–8):1145–54. https://doi.org/10.1111/j.1365-2702.2011.03820.x.

    Article  PubMed  Google Scholar 

  35. Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta‐analyses: The PRISMA statement. BMJ. 2009;339:b2535. https://doi.org/10.1186/2046-4053-4-1.

    Article  PubMed  PubMed Central  Google Scholar 

  36. Schmitt J, Langan S, Stamm T, Williams HC. Harmonizing Outcome Measurements in Eczema Delphi panel. Core outcome domains for controlled trials and clinical recordkeeping in eczema: international multiperspective Delphi consensus process. J Invest Dermatology. 2012;131:623–30.

    Article  Google Scholar 

  37. Tugwell P, Boers M, Brooks P, Simon L, Strand V, Idzerda L. OMERACT: an international initiative to improve outcome measurement in rheumatology. Trials. 2007;8:38. https://doi.org/10.1186/1745-6215-8-38.

    Article  PubMed  PubMed Central  Google Scholar 

  38. Williamson R, Altman D, Blazeby J, Clarke M, Devane D, Gargon E, Tugwell P. Developing core outcome sets for clinical trials: issues to consider. Trials. 2012;13:132. https://doi.org/10.1186/1745-6215-13-132.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

The authors are grateful to all the Mental Health Nurses in practice, NMPDU staff, work-stream Working-Group members, Directors of Mental Health Nursing, Practice Development Coordinators, who contributed to the collaborative development of this suite of quality care process metrics and indicators for mental health nursing practice in Ireland.

Funding

This project was funded by the Office of Nursing and Midwifery Services Directorate, Health Service Executive. The funder was involved, through membership of the steering group, identification of potential participants, study design, data collection, data analysis, and in completion of the manuscript.

Author information

Authors and Affiliations

  1. School of Nursing and Midwifery, National University Ireland Galway, Galway, Ireland

    Andrew Hunter, Nora Barrett, Martina Giltenane & Louise Murphy

  2. Nursing and Midwifery Planning and Development Unit, Health Services Executive, North-West, Dublin, Ireland

    Anne Gallen

  3. Nursing and Midwifery Planning and Development Unit, Health Services Executive, West/Mid-West, Dublin,, Ireland

    Gillian Conway

  4. Nursing and Midwifery Planning and Development Unit, Health Services Executive, North East, Dublin, Ireland

    Anne Brennan

Authors
  1. Andrew Hunter
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nora Barrett
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Anne Gallen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Gillian Conway
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Anne Brennan
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Martina Giltenane
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Louise Murphy
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

AH, NB, AG, and GC contributed to the design of the project. AH, NB and LM conducted the systematic review and the surveys. AH, NB, and LM analysed the data. AH and LM drafted the manuscript. AG, AB, MG, and GC contributed to the draft. All authors reviewed the final draft version of the manuscript prior to submission. The author(s) read and approved the final manuscript.

Corresponding author

Correspondence to Andrew Hunter.

Ethics declarations

Ethics approval and consent to participate

This research was undertaken in accordance with required ethical guidelines and regulations. Accordingly approval for this study was granted by the Research Ethics Committee, National University of Ireland Galway on the 13th of December 2016 [Ref: 16-Dec-09, Mental Health Stream]. Informed consent was obtained from all participants. All participants had to indicate their explicit consent to participate by clicking on an ‘I agree’ button on the online participant information sheet before they could access each round of the Delphi. All potential consensus meeting participants were given a participant information leaflet and written consent to participate was obtained from each willing participant at the meeting.

Consent for publication

Not applicable.

Competing interests

None.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hunter, A., Barrett, N., Gallen, A. et al. Collaborative identification and prioritisation of mental health nursing care process metrics and indicators: a Delphi consensus study. BMC Health Serv Res 22, 350 (2022). https://doi.org/10.1186/s12913-022-07659-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12913-022-07659-2

Keywords

BMC Health Services Research

ISSN: 1472-6963

Contact us