Rethinking workforce planning for integrated care: using scenario analysis to facilitate policy development
BMC Health Services Research volume 20, Article number: 429 (2020)
A goal of health workforce planning is to have the most appropriate workforce available to meet prevailing needs. However, this is a difficult task when considering integrated care, as future workforces may require different numbers, roles and skill mixes than those at present. With this uncertainty and large variations in what constitutes integrated care, current health workforce policy and planning processes are poorly placed to respond. In order to address this issue, we present a scenario-based workforce planning approach.
We propose a novel mixed methods design, incorporating content analysis, scenario methods and scenario analysis through the use of a policy Delphi. The design prescribes that data be gathered from workforce documents and studies that are used to develop scenarios, which are then assessed by a panel of suitably qualified people. Assessment consists of evaluating scenario desirability, feasibility and validity and includes a process for indicating policy development opportunities.
We confirmed our method using data from New Zealand’s Older Persons Health sector and its workforce. Three scenarios resulted, one that reflects a normative direction and two alternatives that reflect key sector workforce drivers and trends. One of these, based on alternative assumptions, was found to be more desirable by the policy Delphi panel. The panel also found a number of favourable policy proposals.
The method shows that through applying techniques that have been developed to accommodate uncertainty, health workforce planning can benefit when confronting issues associated with integrated care. The method contributes to overcoming significant weaknesses of present health workforce planning approaches by identifying a wider range of plausible futures and thematic kernels for policy development. The use of scenarios provides a means to contemplate future situations and provides opportunities for policy rehearsal and reflection.
Health workforce planning seeks to ensure the right people receive the right services at the right place, at the right time, from those with the right skills  and at the right price or cost . This task has become increasingly challenging for workforce planners as health care embraces increasing integration , as a future health workforces’ configurations may not be known , may be quite different from those at present  and may require people to possess different or strengthened competencies . These uncertainties are exacerbated by the current medical education settings, which do not cater well for the teaching of integrated care practices [6, 7].
Adding to this is the problem that integrated care is ill defined, though it is broadly considered to be an inter-sectoral approach that aims to align the health care system with other human service systems . There is an array of definitions of integration and how integrated care is configured, implemented and coordinated, however these have the common aims of improving outcomes for the target population, enhancing their quality of life and improving consumer satisfaction . Moreover, as a developing field, integrated care lacks policy clarity and has few of the hallmarks of systematic policy development and implementation . Thus, the understanding of integrated care and its implementation may require the use of other than traditional research approaches to provide needed insights . In response, we propose a contrasting approach to traditional health workforce planning. Rather than attempting to predict future integrated care workforce roles or numbers, we present a novel method as a means to contribute to a better understanding of how integrated care workforces may evolve and to reveal associated policies.
Thus, the aim of the article is to present and discuss our method. We continue by outlining the planning context, the characteristics of integrated care that act to limit traditional workforce planning approaches, and the narrow use of scenarios in current workforce planning. Next, we detail our mixed methods approach, which acts to reduce uncertainty and to provide improved clarity of workforce structures and policy components. Finally, we outline the results of an application of the method and discuss its utility.
Integrated care in New Zealand
A number of countries have developed strategies or policies intended to promote the integration of care [12, 13]. New Zealand, as one of these, has integrated some of its health planning and funding functions, but has experienced few service improvements . The nation’s integrated care focus is enabled by its Health Strategy, which identifies an “integrated approach” as key to outcomes , p.1, although more effective ways of working and planning have been advised .
A sector that can benefit from such improved service and workforce planning is that of Older Persons Health (OPH), a sub-sector that serves a growing population  with multiple health and social care needs requiring multidisciplinary collaboration and coordination . New Zealand’s rates of people over 65 being in care have steadily fallen since the 1970s to less than 7 % in the mid-2000s and is attributed to a wider acceptance of community living, changes to state residential care subsidy criteria and a broadening of community-based support services . OPH in New Zealand ranges from fully independent living to specialist and secure care facilities  (see Fig. 1), with more than half of those in residential facilities being 85 years or older . A result of the high numbers of older people residing outside of residential care has increased both the importance of the family’s role and the numbers of community based support and care workers . These workers are predominantly female, older, part-time and lowly paid .
Integrated care and health workforce planning
Integrated care has been described as a conceptual “imprecise hodgepodge”, with diverse meanings and multiple interactions between numerous actors, hampering its systematic understanding and successful real-world application , p.12. As such, integration can be orientated: for individuals, as case management or care plans; for groups, as models for chronic care or the frail and elderly; for specific long-term conditions such as diabetes or cardiovascular disease; or for populations . This diversity has policy quality effects, impacts on implementation quality  and adds to stakeholder confusion over integration’s purpose or how to proceed , particularly when inter-professional collaboration is dictated by integrated service designs .
Thus, integrated care and its lack of definitional regularity provides problems for traditional health workforce planning approaches. Mostly, these approaches are based on the matching of worker volumes with expected demand using highly simplistic models that are usually developed from historical allocations of single professions . These models also tend to pay little attention to a population’s health needs  and the variations that may exist within , thereby producing poor results  and perpetuating the health system status quo . Though, while these limitations are well known, few health workforce planning systems are able to cater for the requirements of integrated care . While there are a few approaches that enable planners to review, revise and remodel roles, tasks and worker numbers when designing such services [25, 28, 29], these methods may not be practical over longer time horizons or when it is difficult to quantify role-mix or service innovation impacts .
When faced with uncertainty, such as that posed by integrated care, scenarios are an appropriate method to gain situational understanding or to explore a particular issue . As such they can be used to improve organisational integration, communication and learning , to examine conditions of uncertainty, to understand intractable problems [32, 33], and to formulate strategic responses and decisions . Van der Heijden  notes that scenarios aid decision making in two ways, through interactive rational analysis or through identifying and describing the social interactions and processes, which lead to a future.
Many health workforce planning scenarios reported in the literature tend to be quantitative in nature , applying what-if or predictive scenarios as a means to scout planning dimensions or estimating what is expected to occur in the short term . However another, the qualitative scenario, is also capable of assisting health workforce planning . This approach presents scenarios as descriptive narratives , which may: be normative, where the scenario has explicit, accepted or normative starting points and focuses on how certain future situations or objectives can be realised or what we should do; or, be exploratory, which aim to explore a variety situations or developments that are regarded as possible . While the qualitative scenario features less in the health workforce literature , their use can provide policy makers with perspectives that are “devoid of current constraints, vested interests and current concerns” and with space to consider the important over the urgent or to “focus on what really matters” (, p.4).
The reluctance to use such descriptive scenarios in public policy making has been attributed to scenarios’: broadness, making them difficult for particular policy development ; timescales, which are generally longer than those of the policy maker ; difficulty of merging scenarios with the policy making process ; and their easy dismissal on grounds of credibility, legitimacy, and salience . Moreover, public institutions tend to be intolerant of uncertainty, driven by a belief that predicting the future is reliable . These institutions’ policy makers tend to have an affinity for and a reliance on numerical data, even if its accuracy is questionable  and face political pressure to avoid mistakes . The reluctance is despite the knowledge that to strengthen policy analysis we must explore its methodological challenges  and apply a wider range of methods .
Considering these issues, we developed an approach that allows us to better address workforce planning uncertainties, such as those posed by integrated care, and to uncover policy development possibilities. Before implementing our approach, we applied for and received ethical approval from our host university.
Mixed Methods (MM) is an increasingly common health care research approach that intentionally integrates a number of methods , which provides a reliable and valid way to build on one method’s findings of by applying another, integrating types of data, or embedding one analysis within another . These actions require more involved reporting, with researchers recommended to describe, justify and present their designs, methods and results clearly and transparently to reveal their study’s unique insights [52,53,54].
Our design, presented in Fig. 2, is sequential , where we firstly construct the scenarios by applying scenario methods  with recent workforce thematic and actor (stakeholder) data, followed by scenario analysis, which is the application or use of scenarios by individuals or groups in order to explore a particular issue . For the analysis, we used the structured group communication technique, the policy Delphi, which is appropriate in this context as it seeks to generate the strongest possible opposing views for a policy issue  by forcing participants to think about issue pros and cons .
There is no single way to construct scenarios , though Glenn and The Futures Group International  describe eight approaches that share the common stages of diagnosis, development and reporting . Taking note of these, we took our scenario construction data from two sources. The first being an examination of eight workforce documents that reflect a future state of OPH service delivery and workforce issues (see Table 1). These documents were identified as part of a wider structured grey literature review, which applied targeted searches within government, multinational agencies and national health research organisation websites.
Using the Google site search function (where the term “site:websiteURL” is inserted after the search terms), which limits the search solely to the website selected, we searched New Zealand’s Ministry of Health website. We used the search terms ‘health workforce’, ‘forecasting’ and ‘health labour (labor) market’ and followed this by a full catalogue scan to ascertain that all possible documents had been identified. The selection identified those documents that directly concerned the OPH workforce or which mentioned older people as having conditions, being treated by and/or receiving care from one or more relevant workforces.
Our second source of data was taken from Rees et al. , which provided sufficient data and variations to construct a range of exploratory scenarios . Taking the OPH data from this actor analysis of two New Zealand’s health system sub-sectors, provided us with the identification of critical factors for the future OPH workforce, the relative power of actors within the system and the divisiveness of identified issues. These data indicated which actors hold more power and the potential drivers of change for a future workforce system. As such, these are similar to data produced by a horizon scan, as these identify workforce trends and their implications that the actors saw as critical to the workforce’s future.
We deductively content analysed the eight documents to identify and organised thematic codes . We used a simple form of computer-assisted content analysis, code and retrieve  to extract the data, by applying the search terms ‘vision’, ‘model of care’ and ‘scenario’ as these were expected to provide suitable descriptions of the service’s vision, health needs, configurations, desired models of care and clinical scenarios to provide the objectives and intentions from which to construct a normative scenario . From this process, we derived a range of data in terms of the workforce participants, their cultural, clinical and geographical contexts and service expectations, which together represent the normative or expected trajectory of the sector and indicate how the workforce should be constituted.
We approached the scenario drafting by carefully following step-by-step procedures to plot the narratives using the headline method : a process where headlines are expanded into a narrative utilising the unique combination of conditions provided by the data’s contributing themes. Thus, we took the main themes from the document content analysis and using these as headlines, built a description of how New Zealand’s OPH normative future was likely to develop. By taking the actor analysis’ divisive issues and potential behaviours we identified the alternative scenario main themes  and blended these with the normative to produce two exploratory alternatives. For these we applied a “crisis and response” technique to draft plot lines (, p.72), ensuring distinct differences and responses to reflect the necessary diversity and ambiguity between them .
Once each core narrative was complete, we identified signposts, such as projects or studies that elucidated the scenario’s trajectory (−e.g. the use of specialist nursing or more detailed care planning to improve care outcomes). Each scenario also contained open-ended questions which, with the signposts, are a means to assist readers to engage and connect the scenario to the everyday world . We were careful when constructing the scenarios to keep them parsimonious while including sufficient data for the reader to be able to comprehend and evaluate the described situation. Once the scenarios were complete, we tested them with a small group of academics and practitioners to improve their believability and internal consistency .
Here we employed the policy Delphi, a process where a group of anonymous participants are asked to provide a complete-as-possible picture of the issue under investigation through the group’s diversity and extreme views . While infrequently applied in health compared to the conventional Delphi , the policy Delphi is useful for a context characterised by conflicting interests such as found in health workforce planning .
Panel selection and recruitment
Who participates in a policy Delphi is central to ensuring high quality results . Careful panel selection supports a policy Delphi’s reliability through ensuring appropriate levels of panellist knowledge, experience and understanding of the topic under study [55, 70,71,72]. As the policy Delphi’s aim is to promote varied opinions , a panel’s diversity is just as important . We, therefore, used purposive sampling as our selection method , selecting the panellists “for the important information they can provide” (, p.236) and their “valuable ideas” , p.7. To capture a range of perspectives our sampling strategy followed Okoli and Pawlowski’s  advice by identifying potential panellists from contributors to New Zealand health workforce documents, conference proceedings and New Zealand tertiary education institution websites, who we assessed against the eligibility criteria listed in Table 2.
A policy Delphi’s results depend on the dynamics of its processes rather than statistical power , so large panels may be not be advantageous . Small Delphi panels are able to produce reliable results  when there is sufficient group heterogeneity and knowledge . Consequently, there are no clear rules for panellist numbers , nor directions for setting panel sizes . Lower limit panel suggestions range from five to ten, with upper limit suggestions between twenty and fifty [55, 77, 79, 80]. New Zealand’s small population size, health establishment and two medical schools limited the potential knowledgeable participant population from the start, meaning a smaller panel was more likely to result.
A major problem with Delphi panels is attrition. Sometimes referred to as response or participant fatigue, attrition has been attributed to poor instructions and workload guidance, impersonal communications, question volume and repetition [56, 81, 82]. With the likelihood of a small panel, we were sensitive to this issue, so we included attrition reduction measures within the panellists’ invitations, which included clear instructions, an estimate of workload  and the benefits and practical outcomes . In acknowledgment of time-poor panellists, we also provided pre-constructed scenarios  and pre-tested items  as an effort to reduce the panellists’ engagement time. Never-the-less, Turoff  notes that some panellists will give up regardless.
To recruit the panel, we issued nineteen invitations to a range of practitioners (8), educationalists (8) and policy organisation representatives (3). We judged this number to be sufficient to provide our minimum starting size of ten panellists. The invitation also included an option to refer another person, should the invitee not be able to participate. The invitations resulted in fifteen replies, four of whom declined, and one referral, who also declined, leading to a final panel size of eleven (see Table 3).
To conduct the survey, we chose the forecasting-expert-designed, free-of-charge online survey management web platform, Delphi Decision Aid [80, 86]. Online administration can have positive effects on time, organisation, engagement and data handling [71, 87]. Before proceeding, the panellists received comprehensive instructions on how to access the web-based questionnaires and to answer the questions.
The first round consisted of a two-part questionnaire. The first part of the questionnaire focussed on scenario evaluation. This contained four- or five-point closed questions taken from Turoff’s  scales to judge a policy issue or situation. These questions ask the panellists to grade the scenario’s desirability, from 1 (very undesirable) to 4 (very desirable), probability 1 (very improbable) to 5 (very probable) and confidence in the validity of the scenario’s premises from 1 (unreliable) to 4 (certain). The four-point rating scales contain no middle option as a means to prevent fence sitting, though a no-opinion option was also provided. The second part of the questionnaire required the panellists to answer open-ended engagement questions, to develop policy statements. To derive these statements, we applied an inductive content analysis to identify referential codes. By phrasing these codes as should-type statements, we developed parsimonious policy statements following Martino’s  recommended 25 word maximum and noting their relative importance by recording the code frequency per statement .
From the second round on, the panellists answered another two-part questionnaire, part one re-rating the scenarios and part two rating the policy statements. Policy statement rating was based on Turoff’s four-point scales for desirability and feasibility . These dimensions’ ratings indicate a policy statement’s attractiveness and practicality and are intended to be a means for the respondents to rehearse or visualise the results of the statement’s implied actions.
As feedback is considered to be important to the Delphi process , panellists were provided with descriptive statistics and narrative response summaries following each round . Panellists were also able to review the round responses online. As a question’s response stability was reached, it was eliminated from subsequent rounds progressively acting to reduce the questionnaire size and completion times.
A Delphi generally closes when stability of the panel’s responses is attained . A simple method for measuring a policy Delphi response stability is Scheibe et al.’s  net percentage change approach, which calculates the net percentage change between each round’s data distribution and focusses on the group response as a whole, rather than the variation of individual responses. We chose this method for its ease of use and because as it accommodates non-normal or bi-modal data distributions that can be found in policy Delphi data. Due to the modest panel size, we set the response stability acceptability criterion at 20% as per Nelson , meaning that when the percentage change is less than 20% it is considered to be stable. It may take up to four or five rounds to achieve response stability, as panellist divisions tend to become more polarised in latter rounds .
We must also reveal here that the online website did not always faithfully collect or report the respondent data. Some items returned erroneous results and technical issues with the site were also encountered during questionnaire administration. When corrupt data were returned, the affected question was repeated. This acted to prevent some items reaching the stability threshold and thus provided a few indeterminate results.
Results of content analysis
Using the document search terms, a number of critical concerns were found. These grouped into three thematic areas: (1) the workforce system as it is, (2) desired workforce outcomes and (3) particular workforce issues that are required to be addressed to realise these outcomes. A summary of the content analysis is presented in Table 4.
Results of scenario construction
Following our method, we took Table 4’s data to develop a normative scenario. Into this we blended the workforce actor issues to create alternatives. Two alternatives resulted, representing two diverse development paths, one based on the most divisive issues from the actor analysis, the other based on issues that created less but significant actor disagreement as controversial issues (Table 5). Each scenario was given a title to reflect its representative themes and principal issues (Table 6). The resultant scenarios are provided as an additional file at the end of the article.
Delphi panel progress
Even though our initial panel size was small, we made the decision to remove panellists who had made little or no contribution to the first two rounds as they would have contributed little to the initial data . Question round response rates varied, reflecting participation rates and the effects of panel attrition. We collected commentaries or contacts with panellists to help us understand attrition causes, which indicated that participant time poverty was the main issue.
The panel closed at round four, as through attrition we had reached the minimum recommended panel size (five participants). Nevertheless, by this stage more than half of the scenario and policy statement items had reached response stability.
Results of scenario evaluation
We evaluated our scenarios using the three questions. Of the four items affected by corrupt data, two displayed trends towards the stability threshold, while the other two provided no conclusive probability results (Table 7). This shows that the panel found all of the scenarios to be largely desirable and valid, with one of the alternatives, Care’s Evolution, being seen to be more desirable than the normative, Fit and Functional.
To understand these results’ contexts we further analysed the stable items to determine whether consensus or contention was experienced during the rating process. This analysis revealed that the panellists were in moderate consensus over their rating of the three scenarios and the options presented within them (see Table 8). For those items with contention results we reviewed the panellist comment data. These showed that for Fit and Functional’s validity, the panel was concerned about the sector’s leadership and stakeholder commitment to see this scenario through, while for the desirability of Transitioning Workforces, the scenario was seen as both aspirational and untested, with a review of the scenario’s assumptions suggested.
In general, the panel found that “overall, the scenarios are valid” with the issues being “clearly articulated”, although the scenarios may not have been as “older people centric” as required. Even though there was some general agreement, the panellists’ did not universally endorse the scenarios as written. While Care’s Evolution was the most desirable, being favoured for its “broad health care focus including preventive care and dignity”, and its reliable premises, a panellist noted that “funding adjustment [is] necessary to follow patient needs and shift resources”. Commentaries from the inconclusive probability items for Care’s Evolution and Transitioning Workforces revealed that panellists’ had little confidence in the sector’s stakeholders to make decisions that corresponded with the scenarios’ visions by questioning how the actors would apply their power and influence and if the actors would just act in their own self-interest.
The panellists also commented that the scenarios did not go far enough, with doubts cast over whether these scenarios would meet the needs of populations whose care and wellbeing are not presently well served. Further, integrated care’s definitional variation also may have affected panellist interpretation, with some suggesting that more detail was required.
Results of policy statement development
The inductive analysis of round one’s open-ended questions led to twenty-three policy statements being formed. These encompassed a range of policy issues about the transition from now to the future options concerning leadership, funding, models of care, transitions strategies, the role of the family, the increasing importance of community care workers and sector wages. The frequency of the referential codes for each policy statement, which ranged from fourteen to one, provide an indicator of how the panellists felt about the issue.
As this article is focussed on presenting our approach rather than results and content, we will provide a sample of the policy statement analysis to clarify our approach. Thus, in Table 9 we present four policy statements, selected by code frequency, as examples.
Results of policy statement analysis
We evaluated the selected stable items for consensus or contention by reviewing the item-central tendency statistics, a visual inspection of histograms and a rereading the panel comment data on the ratings to understand panellist rationales (Table 10).
We can see that the stable items 12 and 16 record consensus for their desirability, while item 11’s feasibility is contentious. This contention is derived from panellist differences over whether improved funding will translate into better sector wages. Item 18, the statement directly related to the preferred care model, had the highest frequency of references, but never-the-less it failed to reach stability. Reviewing the statement’s comments, we found that the panel saw this statement’s desirability being related to the existence of “integrated teams” and the presence of “good leadership”, with their feasibility doubts due to context dependency, service complexity and a reliance on inter-sectoral cooperation and capacity as patients move from and between various OPH and primary care services.
The aim of the article is to present and discuss scenarios and their analysis as a novel health workforce planning and policy analysis tool. We achieve this by following van der Heijen’s  option of using scenarios to identify and describe social and professional interactions that lead to an understanding of the processes that lead to a future. In doing this, we are not seeking perfect consensus on an idealised prospect. Rather, we are trying to explore opinions and identify positions that point us to issues that may impede or enhance the reaching of integration goals and, importantly, elicit signals for an understanding of the dynamics and the needs of its workforces. Thus, the data we derive informs us as to how stakeholders may interact and to identify what values and behaviours they would need to express for the described situation to be realised . In a successful networked model of care we would expect to see stakeholders exhibiting mutual self-interest, cross-boundary connectedness and interprofessional support , though we found the panellists questioning how deeply these characteristics are held or expressed by the sector’s actors. This reveals an important aspect of scenario analysis; that it can act as a rehearsal of a policy’s implementation, with the resulting commentaries providing service and workforce planners a further means to identify the policy implications for service outcomes, its workforces and wider stakeholders.
Allowing for this insight is our choice of analysis method. As indicated previously, the policy Delphi allows for a situation or policy to be canvassed widely and the results and their interpretation are supported by techniques that reveal division and explore dissention and do not rely on consensus [55, 90]. This becomes important when delving into why certain scenarios are seen to be more attractive than others or why their validity comes into question. Thus, we can derive a better understanding of why a situation or policy statement is more favourable for some but not others and what cautions need to be exercised before a policy is presented or implemented. Indeed, we found that even for those items that the panellists’ deem important, contention and lack of response stability can occur. Taking his approach therefore complements the numbers-based workforce planning that drives most policy, particularly those to relieve shortages, by reducing the focus from the short-term and urgent and emphasising outcomes .
On many occasions workforce issues are not well reflected in health policy development  nor in a health system’s governance mechanisms . One proposal to improve on this is to promote the commitment of all professionals and sectors in the policy development process . Though often as not, patients are left out of policy or health planning, or, have little influence even when canvassed . This would indicate that better engagement and involvement processes for patients and their families is required when attempting to integrate care and its service design and policy development processes.
However our approach is not without its detractions. We acknowledge the methodological limitations of Delphi, including participant time poverty and panel attrition [81, 95]. Even though we applied many of the Delphi method’s optimisation recommendations, we still experienced limiting issues. The decision to survey online [71, 87] was successful to some degree with reduced survey administration and data handling intervals. Though these did not necessarily improve participation or response times. The online option assumes that people have the requisite skills and introduces additional issues. Our experience suggests that another form of opinion polling or ranking method that also accounts for dissention could be chosen to refine the method and avoid some of the Delphi’s limitations. However, we must remember that any technique will still require a population of respondents, many of whom will be time poor.
There is also the limitation of the scenarios themselves. There is a range of methods to enable scenario creation . Using another method may have produced scenarios that were more or less desirable, probable or valid, or have been more assertive in pushing boundaries. That said, scenarios need to be plausible and a more expansive approach may have produced a scenario too outlandish for the panel to accept and undermined the process .
Similarly, another limitation is the lack of a consistent definition for integrated care. Not having a common understanding of what is being discussed impedes statement formulation from panellist points of view. The scenarios we presented overcame this to some extent, by stating or implying what integration may look like, though some panellists still asked for more detail. However, developing scenarios with competing integrated care definitions could also be a way for policy makers and service planners to present their versions of integrated care by drafting narratives that reveal how patients and the workforce are proposed to interact and how a care model’s processes may operate differently from the present .
Lastly, there is the possible cultural barrier to adoption of our approach. Policy makers tend to be risk-averse and preferring certainty or predictability, hence the lack of widespread use of scenarios in policy making [45, 46]. Despite this, there is an emerging recognition that agencies should seek a diversity of workforce planning tools to overcome uncertainty [39, 96].
In conclusion, we have presented an approach to address workforce planning uncertainty associated with integrated care. Our method goes past the traditional role of numbers-based delineation of workforce planning to identify issues and policy kernels for integrated care workforces. While our method is unique and has only been applied in New Zealand’s workforce planning environment, it shows potential for other health workforce policy and planning systems. Our approach provides an example of how policy makers and planners can use scenario analysis as a tool to identify workforce policies and to rehearse their implications, particularly when attempting to address integrating care. It also contributes to overcoming a significant weakness of health system governance by identifying health workforce-centric polices.
Availability of data and materials
Supporting data are available upon request from the corresponding author.
Health Workforce New Zealand
Older Persons Health
Birch S, Kephart G, Murphy GT, O'Brien-Pallas L, Alder R, MacKenzie A. Health human resources planning and the production of health: development of an extended analytical framework for needs-based health human resources planning. J Public Health Manag Pract. 2009;15(6):S56–61.
Hurst K, Patterson KD. Health and social care workforce planning and development–an overview. Int J Health Care Q Assur. 2014;27(7):562–72.
Barbazza E, Langins M, Kluge H, Tello J. Health workforce governance: processes, tools and actors towards a competent workforce for integrated health services delivery. Health Policy. 2015;119(12):1645–54.
Suter E, Oelke ND, MADdS L, Vliet-Brown CV, Hepp S, Stiphout M, et al. Indicators and measurement tools for health systems integration: results from a knowledge synthesis. Int J Integ Care. 2017;17(6):4 1–17.
Busetto L, Luijkx K, Calciolari S, Ortiz LGG, Vrijhoef HJM. Exploration of workforce changes in integrated chronic care: findings from an interactive and emergent research design. PLoS One. 2017;12(12):e0187468.
Stein K. Developing a competent workforce for integrated health and social care: what does it take? Int J Integr Care. 2016;16(4):9 1–3.
Gauld R. Disrupting the present to build a stronger health workforce for the future: a three-point agenda. J Prim Health Care. 2018;10(1):6–10.
Leutz WN. Five laws for integrating medical and social services: lessons from the United States and the United Kingdom. Milbank Q. 1999;77(1):77–110.
Nolte E, McKee M. Integration and chronic care: a review. In: Nolte E, McKee M, editors. Caring for people with chronic conditions: a health system perspective. Maidenhead: McGraw-Hill Education (UK); 2008. p. 64–91.
Kaehne A, Glasby J, Miller R. Editorial: making sense of integration. J Integr Care. 2018;26(1):2–3.
Wistow G, Dickinson H. Integration: work still in progress. J Health Org Manag. 2012;26(6):676–84.
McMenamin A, Mannion R. Integrated health workforce planning: the key enabler for delivery of integrated care? Int J Integr Care. 2017;17(5):A278 1-8.
Billings J. Developing a whole systems joint outcomes approach to contracting for integrated health and social care for older people. Int J Integr Care. 2016;16(6):A77 1-8.
Cumming J. Integrated care in New Zealand. Int J Integ Care. 2011;11(e138):1–13 Available from: https://ijic.ubiquitypress.com/articles/10.5334/ijic.678/galley/1496/download/.
Minister of Health. New Zealand Health Strategy: Future direction. Wellington: Ministry of Health; 2016.
Ministry of Health. The role of health workforce New Zealand. Wellington: Ministry of Health; 2014.
Associate Minister of Health. Healthy Ageing Strategy. Wellington: Ministry of Health; 2016.
SUSTAIN-project. Lessons learned from improving integrated care in Europe. Bilthoven: Sustainable tailored integrated care for older people in Europe (SUSTAIN-project); 2019. Available from: https://www.sustain-eu.org/wp-content/uploads/sites/4/2019/01/SUSTAIN-overarching-policy-report_final-version.pdf.
Swarbrick N. Care and carers - care of elderly people. 2011 14 November 2016. Available from: http://www.TeAra.govt.nz/en/care-and-carers/page-2.
Ministry of Health. Health of older people workforce. Wellington: Ministry of Health; 2016. [updated 13 July 2016]. Available from: https://www.health.govt.nz/nz-health-statistics/health-statistics-and-data-sets/older-peoples-health-data-and-stats/health-older-people-workforce.
Kodner DL. All together now: a conceptual exploration of integrated care. Healthc Q. 2009;13:6–15.
World Health Organisation. Integrated care models: An overview. Working document. Copenhagen: WHO Regional Office for Europe; 2016.
Manthorpe J, Iliffe S. Joining up dementia: not as easy as it sounds. J Integr Care. 2017;25(4):280–7.
Jolanki O, Tynkkynen L-K, Sinervo T. Professionals’ views on integrated care. J Integr Care. 2017;25(4):247–55.
Segal L, Leach MJ. An evidence-based health workforce model for primary and community care. Implement Sci. 2011;6(93):1–8.
Gorman DF. Developing health care workforces for uncertain futures. Acad Med. 2015;90(4):400–3.
Gorman DF. Matching the production of doctors with national needs. Med Educ. 2018;52(1):103–13.
Segal L, Dalziel K, Bolton T. A work force model to support the adoption of best practice care in chronic diseases–a missing piece in clinical guideline implementation. Implement Sci. 2008;3(35):1–9.
Holmes GM, Morrison M, Pathman DE, Fraher E. The contribution of “plasticity” to modeling how a community’s need for health care services can be met by different configurations of physicians. Acad Med. 2013;88(12):1877–82.
Cairns G, Wright G. Scenario thinking: practical approaches to the future. 2nd ed. Basingstoke: Palgrave Macmillan; 2017.
Wright G, Bradfield R, Cairns G. Does the intuitive logics method - and its recent enhancements - produce "effective" scenarios? Technol Forecasting Soc Change. 2013;80(4):631–42 PubMed PMID: WOS:000317162200006.
Bradfield R, Cairns G, Wright G. Teaching scenario analysis—an action learning pedagogy. Technol Forecast Soc Chang. 2015;100:44–52.
Wright G, Cairns G, O’ Brien F, Goodwin P, et al. Eur J Oper Res. 2019;278(1):3–19.
Van Der Heijden K. Scenarios and forecasting: two perspectives. Technol Forecast Soc Chang. 2000;65(1):31–6.
Rees GH, Gauld R, Crampton P, MacDonell S. The promise of complementarity: using the methods of foresight for health workforce planning. Health Serv Manag Res. 2018;31(2):97–105.
Börjeson L, Höjer M, Dreborg K-H, Ekvall T, Finnveden G. Scenario types and techniques: towards a user's guide. Futures. 2006;38(7):723–39.
Boeckmann M. Quantitative and qualitative forecasting methods in health workforce planning. In: Handbook of Health Workforce Planning Methodologies Across EU Countries, DO52, Release 1. Brussels: Joint Action on Health Workforce Planning & Forecasting; 2015. Available from: http://healthworkforce.eu/wp-content/uploads/2015/11/150306_WP5_D052-Handbook-on-HWF-Planning-Methodologies-across-EU-Countries_Release-1_Final-version.pdf.
Glenn JC. The Futures Group International. Scenarios. In: Futures research methodology - V30. Washington, DC: The Millennium Project; 2009. p. 1–52. Available from: http://www.millennium-project.org/millennium/FRM-V3.html.
Rees GH, Crampton P, Gauld R, MacDonell S. New Zealand's health workforce planning should embrace complexity and uncertainty. N Z Med J. 2018;131(1477):109–15.
World Economic Forum. Sustainable health systems: visions, strategies, critical uncertainties and scenarios. Geneva: World Economic Forum; 2013.
Riddell GA, van Delden H, Dandy GC, Zecchin AC, Maier HR. Enhancing the policy relevance of exploratory scenarios: generic approach and application to disaster risk reduction. Futures. 2018;99:1–15.
Hughes N. Towards improving the relevance of scenarios for public policy questions: a proposed methodological framework for policy relevant low carbon scenarios. Technol Forecast Soc Chang. 2013;80(4):687–98.
Bryson J, Piper J, Rounsevell M. Envisioning futures for climate change policy development: scenarios use in European environmental policy institutions. Environ Policy Gov. 2010;20(5):283–94.
Rickards L, Wiseman J, Edwards T, Biggs C. The problem of fit: scenario planning and climate change adaptation in the public sector. Environ Planning C Govt Pol. 2014;32(4):641–62.
van Asselt MBA, Faas A, van der Molen F, Veenman S. Exploring Futures for Policymaking. The Hague: WRR [Scientific Council for Government Policy]; 2010. Contract No.: 2939.
Veenman SA. Futures studies and uncertainty in public policy: A case study on the ageing population in the Netherlands. Futures. 2013;53:42–52.
Enserink B, Kwakkel JH, Veenman SA. Coping with uncertainty in climate policy making: (Mis) understanding scenario studies. Futures. 2013;53:1–12.
Walt G, Shiffman J, Schneider H, Murray SF, Brugha R, Gilson L. ‘Doing’ health policy analysis: methodological and conceptual reflections and challenges. Health Policy Plann. 2008;23(5):308–17.
Dunn WN. Public policy analysis: an introduction. 3rd ed. Upper Saddle River: Pearson / Prentice Hall; 2004.
Creswell JW, Klassen AC, Plano Clark VL, Clegg SK. Best practices for mixed methods research in the health sciences. Washington DC: National Institutes of Health, Office of Behavioral and Social Science Research; 2011.
Zhang W, Creswell JW. The use of "mixing" procedure of mixed methods in health services research. Med Care. 2013;51(8):e51–e7.
Wisdom JP, Cavaleri MA, Onwuegbuzie AJ, Green CA. Methodological reporting in qualitative, quantitative, and mixed methods health services research articles. Health Serv Res. 2012;47(2):721–45.
O'Cathain A, Murphy E, Nicholl J. Integration and publications as indicators of" yield" from mixed methods studies. J Mixed Methods Res. 2007;1(2):147–63.
O'Cathain A, Murphy E, Nicholl J. The quality of mixed methods studies in health services research. J Health Serv Res Policy. 2008;13(2):92–8.
Turoff M. The policy Delphi. In: The Delphi Method; 2002. p. 80–96. Available from: http://www.is.njit.edu/pubs/delphibook/.
Meskell P, Murphy K, Shaw D, Casey D. Insights into the use and complexities of the policy Delphi technique. Nurse Researcher. 2014;21(3):32–9.
Schwartz P, Olgilvy JA. Plotting your scenarios. In: Fahey L, Randall RM, editors. Learning from the future: competitive foresight scenarios. New York: John Wiley & Sons; 1998. p. 57–80.
Cole S, Chichilnisky G. Modelling with scenarios: Technology in North-South development. Futures. 1978;10(4):303–21.
Gorman DF. Towards a sustainable and fit-for-purpose health workforce - lessons from New Zealand. Med J Aust. 2012;1(Suppl 3):32–6.
Rees GH, Crampton P, Gauld R, MacDonell S. Rethinking health workforce planning: capturing health system social and power interactions through actor analysis. Futures. 2018;99:16–27.
Elo S, Kyngäs H. The qualitative content analysis process. J Adv Nurs. 2008;62(1):107–15.
Maietta RC. Computer-assisted data analysis. In: Given LM, editor. The sage encyclopedia of qualitative research methods. Thousand Oaks: Sage Publications; 2008. p. 103–7.
Coates JF, Glenn JC. Normative forecasting. In: Futures research methodology - V30. Washington, DC: The Millennium Project; 2009. p. 1–39. Available from: http://www.millennium-project.org/millennium/FRM-V3.html.
Franco FL, Canen AG, Pizzolato ND. Strategic alliances: tools for constructing the future. Business Strategy Series. 2011;12(2):84–97.
Schoemaker PJH. Twenty common pitfalls in scenario planning. In: Fahey L, Randall RM, editors. Learning from the future: competitive foresight scenarios. New York: John Wiley & Sons; 1998. p. 422–31.
Willis G, Cave S. Scenario generation: enhancing scenario generation and quantification. London: Centre for Workforce Intelligence; 2014.
Rauch W. The decision Delphi. Technol Forecast Soc Change. 1979;15(3):159–69.
Bijl R. Delphi in a future scenario study on mental health and mental health care. Futures. 1992;24(3):232–50.
Schneider JB. The policy Delphi: A regional planning application. Technol Forecast Soc Change. 1971;3(0):481–97.
Martino JP. Technological forecasting for decision making. 3rd ed. New York: McGraw Hill; 1993.
Franklin KK, Hart JK. Idea generation and exploration: benefits and limitations of the policy Delphi research method. Innov High Educ. 2007;31(4):237–46.
O’Loughlin R, Kelly A. Equity in resource allocation in the Irish health service: a policy Delphi study. Health Policy. 2004;67(3):271–80.
Spickermann A, Zimmermann M, von der Gracht HA. Surface- and deep-level diversity in panel selection — exploring diversity effects on response behaviour in foresight. Technol Forecast Soc Chang. 2014;85:105–20.
Palys T. Purposive sampling. In: Given LM, editor. The sage encyclopedia of qualitative research methods. Thousand Oaks: Sage Publications; 2008. p. 697–8.
Maxwell JA. Designing a qualitative study. In: Rog DJ, Bickman L, editors. The handbook of applied social research methods. 2nd ed. Thousand Oaks: Sage; 2008. p. 214–53.
Gordon TJ. The Delphi method. In: Futures research methodology - V30. Washington, DC: The Millennium Project; 2009. p. 1–31. Available from: http://www.millennium-project.org/millennium/FRM-V3.html.
Okoli C, Pawlowski SD. The Delphi method as a research tool: an example, design considerations and applications. Inf Manag. 2004;42(1):15–29.
Akins RB, Tolson H, Cole BR. Stability of response characteristics of a Delphi panel: application of bootstrap data expansion. BMC Med Res Methodol. 2005;5(37):1–12.
Rowe G, Wright G. Expert opinions in forecasting: the role of the Delphi technique. In: Armstrong JS, editor. Principles of forecasting: a handbook for researchers and practitioners. Boston: Kluwer Academic Publisher; 2001. p. 125–43.
Green K, Armstrong JS, Graefe A. Methods to elicit forecasts from groups: Delphi and prediction markets compared. Foresight. 2007;8:17–20.
Cramer CK, Klasser GD, Epstein JB, Sheps SB. The delphi process in dental research. J Evid Based Dent Pract. 2008;8(4):211–20.
Förster B, von der Gracht H. Assessing Delphi panel composition for strategic foresight - a comparison of panels based on company-internal and external participants. Technol Forecast Soc Chang. 2014;84:215–29.
Scapolo F, Miles I. Eliciting experts' knowledge: A comparison of two methods. Technol Forecast Soc Change. 2006;73(6):679–704 PubMed PMID: WOS:000238473500005.
Rowe G, Wright G. The Delphi technique: Past, present, and future prospects — Introduction to the special issue. Technol Forecast Soc Change. 2011;78(9):1487–90.
Bowman G, MacKay RB, Masrani S, McKiernan P. Storytelling and the scenario process: understanding success and failure. Technol Forecast Soc Chang. 2013;80(4):735–48.
Armstrong JS, Fildes R. Making progress in forecasting. Int J Forecast. 2006;22(3):433–41.
Pincombe B, Blunden S, Pincombe A, Dexter P. Ascertaining a hierarchy of dimensions from time-poor experts: linking tactical vignettes to strategic scenarios. Technol Forecast Soc Change. 2013;80(4):584–98.
Kabanoff B, Holt J. Changes in the espoused values of Australian organizations 1986—1990. J Organ Behav. 1996;17(3):201–19.
Rowe G, Wright G. The impact of task characteristics on the performance of structured group forecasting techniques. Int J Forecasting. 1996;12(1):73–89 PubMed PMID: WOS:A1996UQ69700006.
Linstone HA, Turoff M. Delphi: a brief look backward and forward. Technol Forecast Soc Chang. 2011;78(9):1712–9.
Scheibe M, Skutsch M, Schofer J. Experiments in Delphi Methodology. In: The Delphi Method; 2002. p. 257–81. Available from: http://www.is.njit.edu/pubs/delphibook/.
Ferlie E, Fitzgerald L, McGivern G, Dopson S, Bennett C. Making wicked problems governable? Oxford: Oxford University Press; 2013.
Dussault G, Dubois C-A. Human resources for health policies: a critical component in health policies. Hum Resour Health. 2003;1(1):1.
Hastings SE, Armitage GD, Mallinson S, Jackson K, Suter E. Exploring the relationship between governance mechanisms in healthcare and health workforce outcomes: a systematic review. BMC Health Serv Res. 2014;14(1):479.
Mullen PM. Delphi: myths and reality. J Health Org Manag. 2003;17(1):37–52.
Health and Disability System Review. Health and Disability System Review - Interim Report. In: Hauora Manaaki ki Aotearoa Whānui - Pūrongo mō Tēnei Wā. Wellington: HDSR; 2019.
GR would like to thank the study participants for their time and attention, Prof. Peter Brooks for his time and energy assisting with advice and questionnaire, website and scenario testing and feedback and the attendees of the 2018 European Health Management Association annual conference poster presentation session for their insightful questions and publication suggestions; all of which helped to produce and improve the quality of this work.
The corresponding author received funds from a University of Otago-Health Workforce New Zealand PhD Scholarship and a Freemason Post Graduate Scholarship to assist with completing their PhD.
Ethics approval and consent to participate
Ethical approval was obtained from the University of Otago Ethics Committee reference code D14/260, 2014. Written informed consent was obtained from participants before commencement of the Delphi panel.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Rees, G.H., Crampton, P., Gauld, R. et al. Rethinking workforce planning for integrated care: using scenario analysis to facilitate policy development. BMC Health Serv Res 20, 429 (2020). https://doi.org/10.1186/s12913-020-05304-4