Development and preliminary validation of a leadership competency instrument for existing and emerging allied health professional leaders
BMC Health Services Research volume 16, Article number: 64 (2016)
No instruments, to our knowledge, exist to assess leadership competency in existing and emerging allied health professional (AHP) leaders. This paper describes the development and preliminary exploration of the psychometric properties of a leadership competency instrument for existing and emerging AHP leaders and examines (i) its factor structure, (ii) its convergent validity with the Leadership Practices Inventory (LPI), and (iii) its discriminative validity in AHPs with different grades.
During development, we included 25 items in the AHEAD (Aspiring leaders in Healthcare-Empowering individuals, Achieving excellence, Developing talents) instrument. A cross-sectional study was then conducted in 106 high-potential AHPs from Singapore General Hospital (34 men and 72 women) of different professional grades (49 principal-grade AHPs, 41 senior-grade AHPs, and 16 junior-grade AHPs) who completed both AHEAD and LPI instruments. Exploratory factor analysis was used to test the theoretical structure of AHEAD. Spearman correlation analysis was performed to evaluate the convergent validity of AHEAD with LPI. Using proportional odds regression models, we evaluated the association of grades of AHPs with AHEAD and LPI. To assess discriminative validity, the c-statistics – a measure of discrimination – were derived from these ordinal models.
As theorized, factor analysis suggested a two-factor solution, where “skills” and “values” formed separate factors. Internal consistency of AHEAD was excellent (α-values > 0.88). Total and component AHEAD and LPI scores correlated moderately (Spearman ρ-values, 0.37 to 0.58). The c-index for discriminating between AHP grades was higher for AHEAD than for the LPI (0.76 vs. 0.65).
The factorial structure of AHEAD was generally supported in our study. AHEAD showed convergent validity with the LPI and outperformed the LPI in terms of discriminative validity. These results provide initial evidence for the use of AHEAD to assess leadership competency in AHPs.
Leadership plays a central and critical component in managing patients toward optimal clinical outcomes, and in recent years, there has been an increasing emphasis on the assessment of leadership performance. In turn, the accurate assessment of leadership performance plays a vital role in the development and improvement of leadership in healthcare organizations.
Leadership competencies are defined as leadership skills and behaviors that contribute to superior performance. By using a competency-based approach to leadership, organizations can better identify and develop their next generation of leaders . Reviewing the literature, most studies on leadership  have focused on supervisory leadership of existing leaders and their immediate followers . Few studies, however, have examined the leadership assessment of potential leaders, and to our knowledge, no studies have done so on healthcare professionals.
Because there is no validated and reliable instrument to holistically assess leadership competency in emerging and existing leaders in healthcare, we developed a leadership competency instrument which we have called the AHEAD (Aspiring leaders in Healthcare - Empowering individuals, Achieving excellence, Developing talents) instrument.
Study design and participants
A cross-sectional survey design was used in this study. Data was collected from July 2013 until February 2014 at the Allied Health Division of the Singapore General Hospital. In total, 106 AHPs (34 men and 72 women) of different professional grades (49 principal-grade AHPs, 41 senior-grade AHPs, and 16 junior-grade AHPs) completed both the AHEAD and LPI instruments. These 106 AHPs are come from Dietetics, Medical Social Services, Occupational Therapy, Physiotherapy, Pharmacy, Podiatry and Speech Therapy departments. Department heads selected promising AHPs with the potential for growth and development, as well as existing leaders in their respective department. This study was exempted from review by the SingHealth Centralised Institutional Review Board.
To ensure that AHEAD items are applicable to all within the division, we first conducted item-generation steps through a literature review. This was followed by item-reduction using pretesting and input from discussion groups. The Leadership Practices Inventory  served as an intellectual and structural model during the development of AHEAD.
Item generation and reduction
To identify and generate items appropriate for a leadership competency instrument, we used 2 strategies: We conducted a systematic review of literature [5–8], and obtained input from relevant heads of departments on the competency domains, items and descriptors related to the knowledge, skills and values associated with effective leadership. Shortlisted items and descriptors were then categorized under 2 domains – Values and Skills. All items were rated using a 5-point Likert scale where 1 = not ready, 2 = somewhat ready, 3 = usually ready, 4 = often ready, and 5 = always ready. The instrument was then pretested with a group of 10 AHPs from 4 departments to examine item relevance and question clarity. Based on pretest findings, the total number of items was reduced from 43 to the final 25 items (Additional file 1).
The aim of this phase was to examine the factorial, convergent, and discriminative validity of the AHEAD instrument. To this end, each participant completed both the AHEAD and the LPI instruments through self-evaluation, followed by a facilitated discussion with their supervisor to reach an agreed score. In addition, participants were provided a resource guide which contained descriptors and suggested developmental resources. The subsequent scores were then sent to the team for collation.(Additional file 2)
In our study, factorial validity was examined by assessing the extent to which the items under “skills” and “values” conformed to their respective theorized constructs . Cross-sectional convergent validity was examined by assessing the extent to which AHEAD was associated with the LPI (described later). Discriminative validity – the ability of an instrument to discriminate between known groups, was examined by assessing the extent to which AHEAD discriminated AHPs of different professional grades – namely, in ascending order, junior, senior, principal, and senior principal grades.
Leadership practices inventory
Cross-sectional convergent validity was examined by assessing the extent to which AHEAD was associated with Kouzes and Posner’s Leadership Practices Inventory (LPI) . The LPI consists of two instruments; the LPI-Observer for the subordinates of the individual that is being assessed to report on their supervisor, and the LPI-Self for the individual or supervisor to self-report. Due to its similarity in assessment format with the AHEAD instrument, only the LPI-Self instrument was used. LPI-Self contains 30 questions covering 5 dimensions of the LPI (Challenging, Inspiring, Enabling, Modeling, and Encouraging). Responses are made on a 5-point Likert-type scale, and a total score ranging from 30 to 150. A higher score would indicate stronger self-perceived leadership skills. Internal reliability coefficients for the LPI-Self ranged between .75 and .87 (mean α = .81) [10, 11].
We used descriptive statistics to characterize the study sample: we used means with SDs and medians with IQRs for continuous variables and frequencies with percentages for categorical variables. To examine the factorial validity of AHEAD, we used exploratory factor analysis (EFA). Given that no prior analyses have been performed on AHEAD, EFA is an appropriate first-step technique to explore the nature and number of factors that account for the covariation between the items . We performed EFA on the polychoric correlation matrix of the ordinal-scaled AHEAD items, and we used ordinary least squares extraction with oblique (oblimin) rotation. Of note, an oblique rotation was used because we expected “Values” and “Skills” to be correlated. To determine the number of factors to be extracted, we relied on 3 principles: (i) inspection of the scree plot, (ii) interpretability of the factors, and (iii) results from Horn’s parallel analysis which uses the “mean eigenvalue criterion” [13–15]. Finally, we examined the internal consistency of each factor using Cronbach’s alpha [9, 16].
To examine convergent validity, we computed Spearman correlation between AHEAD and LPI. To examine the discriminative validity of AHEAD and LPI in differentiating AHPs of different professional grades – an ordinal outcome with 4 categories – we computed the rank-biserial correlations which described the correlations of AHP grades with AHEAD and LPI. To statistically compare discriminative validity of AHEAD and LPI, we used separate proportional odds models and regressed AHP grades on AHEAD and LPI . From each ordinal model, we computed the concordance index (c-index), where a value of 1 represents perfect discrimination and 0.5 represents chance discrimination. Next, we used a bootstrapping procedure (1000 repetitions) to calculate the difference in c-indices between AHEAD and LPI models. Specifically, by subtracting the c-index of the LPI from that of AHEAD, positive values indicated that the discriminative validity index of AHEAD was superior to those of LPI.
Table 1 shows the characteristics of participants who were involved in the AHEAD instrument validation phase. The participants were predominantly female (68 %) and had, on average, 12 years of working experience. Amongst the 106 participants, 16 (15 %) were junior AHPs, 41 (39 %) were senior AHPs; 37 (36 %) were principal AHP; and 12 (12 %) were senior principal AHPs.
Table 2 shows the results of the EFA. EFA and parallel analyses suggested a two-factor solution which explained 49 % of the total variance. Consistent with the original instrument and theory, we named the two factors “Values” and “Skills.” Overall, all AHEAD items had a minimum factor loading of 0.40 and 4 items showed cross loadings (that is, factor loadings exceeded 0.30 on both factors). Contrary to our original hypothesized model, 4 items did not load on their designated factors – namely, items Value 4 (“Courage”), Value 7 (“Inspiration”), Value 9 (“Perseverance”), and Skill 11 (“Interpersonal Skills”). Cronbach alpha values for “Values” and “Skills” factors were 0.88 (95 % CI, 0.88 to 0.95) and 0.93 (95%CI, 0.90 to 0.97), respectively, indicating excellent internal consistency. Cronbach alpha value for the whole AHEAD instrument was 0.95 (95 % CI, 0.92 to 0.97), indicating that it is appropriate to compute a total score for AHEAD.
Table 3 shows the correlations among the component and total measures of AHEAD and LPI. Convergent correlation coefficients between the two instruments were on the order of 0.37 to 0.57, and were statistically significant. As regards discriminative validity, with the exception of the LPI-Enable scores, rank-biserial correlations revealed weak associations between AHP grades and LPI scores (rho values, 0.19 to 0.31). In contrast, rank-biserial correlations between AHP grades and AHEAD scores were stronger and moderate in magnitude (rho values, 0.47 to 0.55, P-values < 0.001). Fig. 1 shows the distribution of AHEAD scores across the different AHP grades. Proportional odds regression further supported the discriminative validity of the AHEAD instrument. Specifically, the c-index for discriminating AHP grades was higher for the total AHEAD scores than for the total LPI scores (0.76 vs. 0.65; bootstrap 95%CI for difference, 0.04 to 0.18). Fig. 2 summarizes the differences (and bootstrap 95%CI) in c-indices between the total AHEAD scores and the LPI (component and total) scores.
In a sample of 106 emerging and existing AHP leaders from 7 professions, the factorial structure of the AHEAD instrument was generally supported. AHEAD showed convergent validity with the LPI. It also showed better validity for discriminating AHP grades than did the LPI. To our knowledge, AHEAD is the first validated specific instrument to measure leadership competency in existing and emerging AHP leaders.
Consistent with our expectations, EFA of the AHEAD items identified a 2-factor solution where “skills” and “values” formed separate factors. Internal consistency of AHEAD (overall and its 2 factors) was excellent and all items loaded significantly (factor loadings exceeded 0.40). Nevertheless, 4 items loaded differently from their designated factors (Table 2). Specifically, “values” items “courage”, “inspiration”, and “perseverance” loaded highly and specifically on the “skills” factor whilst “skills” item “interpersonal skills” loaded moderately on “values” factor.
How do we explain these discrepant results? Reviewing the literature, a few themes emerged. Our findings regarding item “courage” reveal that the circumstances in which courage is displayed in the workplace may have influenced how our participants have rated it. Indeed, business research has shown that courageous action involves calculated risk-taking and these decisions require careful deliberation and preparation ; hence, courage may be less of a pre-existing personal trait  but more of a skill that could be “learned, practiced, and grown alongside our responsibilities” .
As regards item “inspiration”, Bass  has emphasized that inspirational leadership focuses on communicating a compelling vision for the team, expressing confidence in team members, and energizing the team. Specifically, by communicating a vision, inspirational leaders can reinforce the common goals of the team; by expressing confidence in group members, they can enhance the group’s distinctiveness and prestige; and by energizing group members, they can encourage more interpersonal interaction among team members . Thus, the ability to inspire depends not only on the leader’s charisma, but also the leader’s communication and implementation skills [23–25]. Also, given these skills are also more salient and measureable than one’s intrinsic character, it could be argued that inspiration may be skill-based.
Likewise, perseverance could be considered a skill because it depends on several factors that can be learnt. Research shows that grit, a function of perseverance and passion, is associated with educational attainment and age . This association indicates that perseverance may increase with experience and hence, perseverance is not wholly intrinsic in an individual. Furthermore, the authors suggest that perseverance depends not only on motivation, commitment, or zeal, but it also depends on the capacity for hard work .
Our findings regarding “interpersonal skills” show that empathy wields a strong influence on interpersonal skills [27–29] and that the lack of interpersonal skills, or in other words, antisocial behavior – can be characterized by a lack of empathy and a possible inability to understand the emotions of others . And yet, empathy is a significant factor associated with clinical competence and patient outcomes  and crucial for social functioning because interpreting such signals allows one to evaluate when and how to avoid social conflict . In our study, the “interpersonal skills” item loaded moderately on both “skills” and “values” factors (Table 2), and this finding seems to agree with a qualitative study conducted on physicians which reported that empathy were considered both a personality characteristic and a skill .
Convergent and discriminative validity
In our study, moderate correlations were demonstrated between AHEAD and LPI scores (Table 3). To the extent that the LPI is a well-established and validated measure of leadership competency , these correlation results suggest that AHEAD was assessing a similar underlying construct as the LPI, thereby providing evidence for its convergent validity. More importantly, AHEAD outperformed LPI in terms of discriminative validity (Fig. 1).
AHEAD scores may be more discriminative because given that the instrument was developed by AHPs, its conceptual focus may be more relevant to the AHP compared with the LPI. Furthermore, having to justify AHEAD ratings by citing examples may require participants to give a more accurate and honest view of themselves. In contrast, the LPI may be less favourable as a discriminative measure because the comparatively simpler way of rating oneself with LPI may influence respondents to rate themselves more hastily which may have introduced some discrepancy in results. This is especially so for junior AHPs, who may have rated themselves more highly than they would have if they had to produce supporting evidence. Finally, the AHEAD scores may be more discriminative because individual items descriptors were provided in the assessment form whereas only domain descriptors were available for the LPI. Conceivably, the detailed AHEAD item descriptors may bring respondents and their supervisors to a common level of understanding, which in turn, would have enhanced the predictive accuracy of the AHEAD scores.
Our study has limitations. First, our sample size was modest but this limitation may be countered by the few number of factors that we examined, the relatively large number of items per factor, and the moderately-high factor loadings . Thus, based on the simulation results by Mundfrom et al. , we believe our sample size was at least adequate. Second, our focus on AHP leaders may limit study generalizability to this subset of AHPs. However, by limiting our study to AHPs who had the potential to lead and excel, we have provided a more robust test of convergent and discriminative validity of the AHEAD instrument. Third, we studied only AHPs so it remains uncertain if our findings are applicable to other healthcare professionals such as nurses, physicians, and hospital administrators. This limitation must, however, be balanced against the fact that the AHEAD instrument is one of the few leadership competency instruments developed in and for healthcare professionals. Accordingly, future studies should evaluate whether AHEAD could be used (or modified for use) in other healthcare disciplines or settings, preferably adopting a confirmatory factor analysis approach. Fourth, this cross-sectional study focused on convergent and discriminative validity of AHEAD; hence, it would be informative to evaluate, in a longitudinal fashion, the predictive validity of AHEAD scores with future important leadership outcomes – for example, leadership self-efficacy and follower outcomes and satisfaction .
In conclusion, we have developed and validated the AHEAD instrument to evaluate leadership competency in existing and emerging AHP leaders. Although needing further validation, it is our hope that AHEAD will not only be used as an assessment tool, but it will also play a key role in an AHP’s leadership development.
Aspiring leaders in Healthcare - Empowering individuals, Achieving excellence, Developing talents
Allied Health Professional
Leadership practices inventory
Exploratory factor analysis
Brownell J, Goldsmith M. Commentary on “meeting the competency needs of global leaders: A partnership approach”: An executive coach's perspective. Hum Resour Manag. 2006;45(3):309–36.
Bass BM, Stogdill RM. Handbook of Leadership: Theory, Research & Managerial Applications. 3rd ed. New York, NY: The Free Press; 1990.
House RJ, Aditya RN. The social scientific study of leadership: Quo vadis? J Manag. 1997;23(3):409–73.
Kouzes JM, Posner BZ. The Leadership Practices Inventory (LPI): Participant’s Workbook. Volume 47. Hoboken: John Wiley & Sons; 2003.
Brungardt C. The intersection between soft skill development and leadership education. J Leadership Educ. 2011;10(1):1–22.
Hawley SR, Romain TS, Rempel SL, Orr SA, Molgaard CA. Generating social capital through public health leadership training: a six-year assessment. Health Educ Res. 2012;27(4):671–9.
Cacioppe R. An integrated model and approach for the design of effective leadership development programs. Leader Org Dev J. 1998;19(1):44–53.
Popper M, Lipshitz R. Putting leadership theory to work: a conceptual framework for theory-based leadership development. Leader Org Dev J. 1993;14(7):23–7.
Streiner DL, Norman GR. Health measurement scales: a practical guide to their development and use. 3rd ed. New York: Oxford Univ Pr; 2003.
Posner BZ, Kouzes JM. Psychometric properties of the leadership practices inventory-updated. Educ Psychol Meas. 1993;53(1):191–9.
Kouzes JM, Posner BZ: The leadership practices inventory: Theory and evidence behind the five practices of exemplary leaders. Unpublished document Retrieved February 2002, 12:2010.
Stevens J. Applied multivariate statistics for the social sciences. 3rd ed. Mahwah: Erlbaum; 1996.
Garrido LE, Abad FJ, Ponsoda V. A new look at Horn’s parallel analysis with ordinal variables. Psychol Methods. 2013;18(4):454–74.
Horn JL. A rationale and test for the number of factors in factor analysis. Psychometrika. 1965;30(2):179–85.
Velicer WF, Eaton CA, Fava JL. Construct explication through factor or component analysis: A review and evaluation of alternative procedures for determining the number of factors or components. In: Problems and solutions in human assessment. Berlin: Springer; 2000. p. 41–71.
Cronbach LJ. Coefficient alpha and the internal structure of tests. Psychometrika. 1951;16(3):297–334.
Harrell Jr FE. Regression modeling strategies: with applications to linear models, logistic regression, and survival analysis. New York: Springer; 2015.
Revelle W. psych: Procedures for Psychological, Psychometric, and Personality Research. R package version 1.5-8.http://CRAN.R-project.org/package=psych. 2015.
Harrell Jr FE. rms: Regression Modeling Strategies. R package version 4.2-1. http://CRAN.R-project.org/package=rms. 2014.
Reardon KK. Courage as a skill. Harv Bus Rev. 2007;85(1):58–64.
Kerfoot K. The culture of courage. Nurs Econ. 1998;17(4):238–9.
Comer DR, Vega G. Moral courage in organizations: Doing the right thing at work. Armonk: ME Sharpe; 2011.
Bass BM. Leadership and performance beyond expectations. New York: Free Press; 1985.
Joshi A, Lazarova MB, Liao H. Getting everyone on board: The role of inspirational leadership in geographically dispersed teams. Organ Sci. 2009;20(1):240–52.
Ericsson KA, Prietula MJ, Cokely ET. The making of an expert. Harv Bus Rev. 2007;85(7/8):114.
Duckworth AL, Peterson C, Matthews MD, Kelly DR. Grit: perseverance and passion for long-term goals. J Pers Soc Psychol. 2007;92(6):1087.
Graziano WG, Habashi MM, Sheese BE, Tobin RM. Agreeableness, empathy, and helping: a person-situation perspective. J Pers Soc Psychol. 2007;93(4):583.
Batson CD, Early S, Salvarani G. Perspective taking: Imagining how another feels versus imaging how you would feel. Personal Soc Psychol Bull. 1997;23(7):751–8.
Millon T, Simonsen E, Birket-Smith M, Davis RD. Psychopathy: Antisocial, Criminal an Violent Behaviour. New York: Guilford; 1998.
Hojat M, Louis DZ, Markham FW, Wender R, Rabinowitz C, Gonnella JS. Physicians’ empathy and clinical outcomes for diabetic patients. Acad Med. 2011;86(3):359–64.
Shapiro J. How do physicians teach empathy in the primary care setting? Acad Med. 2002;77(4):323–8.
Mundfrom DJ, Shaw DG, Ke TL. Minimum sample size recommendations for conducting factor analyses. Int J Test. 2005;5(2):159–68.
Seibert SE, Silver SR, Randolph WA. Taking empowerment to the next level: A multiple-level model of empowerment, performance, and satisfaction. Acad Manag J. 2004;47(3):332–49.
The authors wish to thank Jacqueline Ong Kia Geok and Jason Ling for their assistance in the development and implementation of the AHEAD instrument, and Winnie Goh from Singapore General Hospital Allied Health Division for her assistance with research. We also thank the various Heads of Department and study participants for their support and cooperation.
This publication was supported in part by the SGH Centre Grant from the National Medical Research Council, Singapore (ref: NMRC/CG/016/2013).
The authors declare that they have no competing interests.
AHG – conception, design, revising manuscript, final approval. KMYJ – acquisition of data, analysis and interpretation of data, drafting of manuscript, revising manuscript, final approval. LJ – conception, design, revising manuscript, final approval. YHP – analysis and interpretation of data, revising manuscript, final approval.
About this article
Cite this article
Ang, HG., Koh, J.MY., Lee, J. et al. Development and preliminary validation of a leadership competency instrument for existing and emerging allied health professional leaders. BMC Health Serv Res 16, 64 (2016). https://doi.org/10.1186/s12913-016-1301-1