Communities of practice and social-professional networks are generally considered to enhance workplace experience and enable organizational success. However, despite the remarkable growth in interest in the role of collaborating structures in a range of industries, there is a paucity of empirical research to support this view. Nor is there a convincing model for their systematic evaluation, despite the significant potential benefits in answering the core question: how well do groups of professionals work together and how could they be organised to work together more effectively? This research project will produce a rigorous evaluation methodology and deliver supporting tools for the benefit of researchers, policymakers, practitioners and consumers within the health system and other sectors. Given the prevalence and importance of communities of practice and social networks, and the extent of investments in them, this project represents a scientific innovation of national and international significance.
Methods and design
Working in four conceptual phases the project will employ a combination of qualitative and quantitative methods to develop, design, field-test, refine and finalise an evaluation framework. Once available the framework will be used to evaluate simulated, and then later existing, health care communities of practice and social-professional networks to assess their effectiveness in achieving desired outcomes. Peak stakeholder groups have agreed to involve a wide range of members and participant organisations, and will facilitate access to various policy, managerial and clinical networks.
Given its scope and size, the project represents a valuable opportunity to achieve breakthroughs at two levels; firstly, by introducing novel and innovative aims and methods into the social research process and, secondly, through the resulting evaluation framework and tools. We anticipate valuable outcomes in the improved understanding of organisational performance and delivery of care. The project's wider appeal lies in transferring this understanding to other health jurisdictions and to other industries and sectors, both nationally and internationally. This means not merely publishing the results, but contextually interpreting them, and translating them to advance the knowledge base and enable widespread institutional and organisational application.
Scholars within the health , aviation , manufacturing , finance , education  and military  sectors have argued that collaborating communities and linked, professionalised networks are significant determinants of important outcomes such as work satisfaction , motivation , recruitment and retention , high performance , organisational resilience , safer organisations  and systems renewal . However, there is a paucity of empirical research, as opposed to normative claims in the literature, to support these arguments. Given the substantial resources invested in supporting collaborating structures, the absence of evidence of their effectiveness is a significant problem. This is especially so given the potential benefits of optimizing the performance of large, complex sectors, such as the health system, which is the specific focus of our project.
Communities of practice can be defined as "groups of people who share a concern, a set of problems, or passion about a topic, and who deepen their knowledge and expertise in this area by interacting on an ongoing basis" . Social networks are "set [s] of people ... ['actors'] ... with some pattern of interactions or 'ties' between them ... [eg] friendships among a group of individuals, business relationships between companies" . In one sense they are two sides of the same coin, but communities of practice scholars have tended to emphasise learning processes while social network scholars have focused on underlying structural properties.
These kinds of clustered relationships abound in health care and other industries. They can be naturally-occurring [eg, groups of clinicians who share professional responsibilities, refer patients to each other, jointly manage care and collaborate over complex cases] or mandated [eg, purpose-designed, sponsored, initiated and funded by organisational leaders] . They can also be tangible and geographically-anchored [eg, in one section of an organisation such as a teaching hospital or general practice] or virtual and geographically-dispersed [eg, international colleagues with common interests linked via telecommunications]. Terms for related phenomena include teams , sub-cultures , micro-systems , inter-professional practices  and integrated services .
Communities of practice [14, 22] and social-professional networks [23, 24] are increasingly seen as crucial determinants in understanding and enhancing group-oriented services. Most literature supports a position that, although there is much to understand, services to patients [clients, customers, purchasers or consumers in other industries] and workplace cultures can be expected to improve where communities of practice and social-professional networks are emphasised and strengthened, whether these are naturally-occurring and emergent, or mandated and purpose-designed networks . These tend to be theoretical or conceptual claims, however, rather than concerted empirical demonstrations.
At the same time, potentially negative consequences have been noted. Wenger and colleagues argue that communities of practice can encourage the hoarding of information to the detriment of others, curtail improvement and progress, and create in-group and out-group rivalries . Buchanan suggests that networks can be a negative as well as positive force, and can mean all the eggs are in one basket. For instance, if a network [eg, electricity grid, computer system, or clinical collaboration] fails, deleterious consequences can result . Communities of practice and social-professional networks can marginalise others, create factions and silos, become vehicles for elitism or isolationism, strengthen us-and-them behaviours and impede organisational sharing [14, 22–31]. Thus we should not accept that communities of practice and social-professional networks are universally useful.
Although interest in communities of practice and social networks in industry settings has grown remarkably, no convincing model for their systematic evaluation has eventuated. Many authoritative commentators agree that analytic evaluation methods and tools are lacking [32–34]. Thus, there is strong consensus amongst scholars that it is vital to develop an appropriate evaluation framework and model with supporting data-gathering tools. If we are to be confident that communities of practice and social networks enable organisational success, and to understand how and to what extent they do so, we need to be able to quantify their contribution to satisfaction, motivation, recruitment and retention, performance, resilience, safety and renewal. Conversely, if these kinds of collaborative structures prove less useful than expected, we need to understand their limitations.
The case for the project
We reviewed the literature on communities of practice and social-professional networks. Significant researchable characteristics of communities of practice include how learning is enabled and constrained, what can be achieved by actively cultivating them, their relationships to organisational learning, interactivity amongst participants within and between communities of practice, how members join and are socialised, the mechanisms of participation, and the stages of their development [14, 22, 28–30]. The literature indicates that the core features of social-professional networks are their deep structure, how they enable change, how information is facilitated by and in them, the processes of information distribution and search, the circumstances in which phase transitions occur, connectedness of hubs and nodes and the conditions which dictate their robustness or fragility [23–27]. In our review of social-professional networks, we applied some of these central concepts to key patient safety issues . We concluded that "Clinicians, like other professionals, work best when they are allowed to flourish in groupings of their own interests and preference, are empowered ... and nurtured and influenced by their peers rather than controlled by others" .
We also conducted a comprehensive analysis of the communities of practice literature pertaining to the health system . We examined the Medline, CINAHL and Embase databases and performed a snowballing search for additional literature. We identified 624 references of which 90 were research papers. Content analysis of the 624 publications via Leximancer, a data mining tool, allowed us to construct Figure 1 which maps the overarching themes in the literature - the social and professional nature of communities of practice, and their strong relationship to service provision and treatment. There are five major themes in the 90 research papers: organisational change [eg, how communities of practice promote productive change, and how change effects communities of practice]; the impact of technology on communities of practice; the contribution of communities of practice to professional and inter-professional learning and the shaping of organisational knowledge; learning processes within communities of practice, including mechanisms of collaboration, structured reflection and the development of shared competencies; and barriers to participation in communities of practice, particularly mono-professional, organisational, institutional and service boundaries and silos. We concluded our review by synthesising the literature as follows: "The systematic analysis reveals that the community of practice idea has been empirically applied in the health sector in useful ways, but research is sporadic, and many studies are descriptive and centred on one profession or location. The transferability and generalisability of findings is limited. A difficulty with the empirical studies is ... a lack of specificity in their claims. For example, at times the argument is made that team work has improved but what 'good team work' means is not clarified. There is scope for more far-reaching, rigorous and systemic research" . This background work leads to the current protocol.
This project's specific aims are to develop and design, test in the field, and then refine, simulate and apply a framework, model and tools which can be used to evaluate communities of practice and social networks for their effectiveness and sustainability. We will work in the health system for compelling reasons. As a large, diverse industry (9.3% of GDP) the health system supports many types of policy, managerial and clinical communities of practice and social-professional networks comprising diverse occupational groups, and traverses the public and private sectors. These characteristics and complexities mirror other key industries, greatly enhancing the potential to transfer findings, theories and project outcomes beyond the health sector.
The project thus more broadly aims to realise a scientific innovation of importance. Without an evaluation framework, model and tools, sponsors will continue to be uninformed as to what communities of practice and social networks deliver; will be unable to identify and measure their strengths and weaknesses; and will be frustrated in their efforts to achieve desirable outcomes, and returns on investment. This has profound implications for how communities of practice and social-professional networks are enabled, configured and resourced. Such research is at the heart of a crucial question: how well do groups of professionals work together and how could they be organised to work together more effectively?
The research strategy and process are novel and innovative and are designed to produce original methods in the building of a new framework, model and tools. These include: the engagement of influential stakeholders and consumer groups; theoretical and empirical advancement; a sophisticated new research approach for doing future work; an innovative, tested, refined framework, model and tools for researching and evaluating communities of practice and social-professional networks; and the gathering of original data from the field using the tools to illuminate progress. Our data gathering tools [questionnaires, interview schedules, case study designs, focus group processes, critical incident mapping protocols] will employ new approaches.
We will inform our work using theories on teams , inter-professional learning , network structures [25, 38], change  and group dynamics . These are fundamental concepts to our enquiry, guiding our thinking, particularly in the initial and model-building phases. We will, however, mainly inform our project using the empirical literature to rigorously build our research designs on the basis of prior research results. Few research projects are both empirically and theoretically multi-modal in the one study.
Methods and design
This is a comprehensive effort to execute a rigorous evaluation methodology, and deliver tools for the benefit of researchers, policymakers, practitioners and consumers. We have the opportunity to achieve both meaningful and transferable advances in the knowledge base of the discipline and to incorporate novel and innovative aims and concepts into the project design. Through the support of key stakeholders within the Australian health system, we will have access to a wide range of workplace communities and networks, including hundreds of clinicians, to test and refine our models. The overarching aims, phases, objectives, methods and outcomes in the approach are depicted in Figure 2. Ethics approval for the study was granted by the University of New South Wales' Human Research Ethics Committee, application number 09085. Informed consent from participants is a key component of the approval.
The methodology is conceptualised in four major phases, matched to the aims and objectives and commensurate with the research approach. Each phase has two components, and involves a little over a year's work to achieve the project's objectives.
Phase 1: development and design phase
The project will produce an evaluation framework founded on empirically-grounded evaluation principles. Initially we will drill further into the literature and derive from this assessment a set of fundamental, working evaluation principles. Work feeding into this process includes seminal contributions by Wenger and colleagues [14, 22, 30] and Watts , Strogatz  and Barabási  as well as our work in health systems social structures and change [42–44].
Secondly, armed with the initial framework, we will design a preliminary evaluation model. There are many evaluation types , but our previous research suggests that a triangulated, multi-method model is best suited to this problem. Our earlier work on the evaluation of education processes [44, 46], information management  and information and communication technologies  demonstrates this kind of model can be comprehensive, have high utility and yield robust data. This phase includes the design of data-gathering tools [including questionnaires, interview schedules, case study protocols for video ethnographic work, critical incident maps, and focus group question schedules] to make assessments of communities of practice and social networks, taking into account recent designs [eg, Verburg and Andriessen's] . A key step is the identification of outcome measures to assess the effectiveness of communities of practice and social-professional networks. We have identified indicators for work satisfaction (5 indicators) , motivation (6) , recruitment and retention (5) , high performance (18) , organisational resilience (5) , safer organisations (7)  and systems renewal (11) [54, 55], totalling 57 indicators against which metrics are designated.
Phase 2: testing and refinement phase
Thirdly, in phase 2, we aim to test the model and tools across a focused sample of communities of practice and social-professional networks. Peak Australian stakeholder groups have agreed to involve a wide range of members and participant organisations in this study. These include networks or communities of clinicians identified by the National Institute of Clinical Studies, accreditation surveyors of the Australian Council on Healthcare Standards, managerial members of the Australian College of Health Service Executives and clinical, managerial and policy groups associated with the Clinical Excellence Commission. We will identify, in conjunction with these stakeholder partners, representative samples of differing types of communities of practices and social-professional networks (naturally-occurring, mandated, geographically-anchored, geographically dispersed) and enrol them in our study. We aim to make comparisons between groups and across group types.
The testing component will yield experience which will, fourthly, enable us to refine the tools, including their validity and reliability. Differing approaches to examine validity and reliability will be taken. Work on the reliability of social network data is important, and optimum outcomes can be achieved by re-applying social network questionnaires and comparing consistency of results [56–58]. We have validated questionnaires and used multi-method approaches to corroborate discrete methods convergently [44, 46–48, 59].
Phase 3: simulation and prediction phase
Phase 3 involves two components: fifthly, simulating the systems dynamics  of communities of practice and social networks and sixthly, testing the models' capacity for prediction . Communities of practice and social networks are complex adaptive systems within larger complex adaptive systems. They can be modelled via simulation tools that facilitate the mapping of variables within and external to them. This enables us to understand relationships between parts of the system and to run predictive simulations. Governing features of complex systems include feedback, processes, resources, accumulation of flows into stocks and time delays . In the case of a community of practice of emergency clinicians, for example, key variables include levels and training of staff, workforce casualisation, delays in processes, incentives and disincentives, and factors which enable and constrain behaviours and attitudes. In this phase we will also conduct controlled simulation experiments. We will develop role plays [n = 20] of case scenarios and test in the laboratory the dimensions of role behaviours and decision-making processes under controlled conditions. Theorised barriers and enablers will be tested in idealised circumstances in this way to analyse the scope of the boundaries and assess hypotheses and predictions.
Phase 4: application and outcome phase
Having derived the refined evaluation framework, model and tools we will seventhly apply them on a large scale to a sample of communities of practice and social networks identified across Australia, covering the public and private sectors, in order to achieve a sample broadly representative of the health care system. In this phase we will gather sufficient data on communities of practice and social-professional networks to assess their value, and judge the extent to which they contribute to work satisfaction, motivation, recruitment and retention, performance, resilience, safety and renewal. We will test the results and calibrate them against the predictive simulation data drawn from phase 3.
This leads to our eighth component - a review of the research outcomes to interpret results and make final observations on the findings. We will use an arm's length, expert panel to triangulate judgements. Such a process further strengthens project outcomes, closing the feedback loop and helping to determine whether communities of practice and social-professional networks realise the results intended for them.
This protocol provides a conceptual framework and research design for evaluating hard-to-research social science manifestations which are key determinants of health systems performance. A multi-methods, phased research design is important and innovative, and has been matched to the phenomena under investigation. This represents a bold approach to creating and then testing an evaluation model.
Study strengths and limitations
Investigating and evaluating complex organisational constructs such as communities of practice and social-professional networks is challenging. Conducting research in naturalistic settings, examining behavioural and attitudinal factors in context, is a key strength. The research design recognises this, and applies a complex systems approach to the examination of complex systems phenomena. Repeated, staged testing of progress with the model introduces validity, reliability and rigour. Incorporating measures of organisational performance in the design increases the potential for robust correlations. Having undertaken extensive literature analyses of both communities of practice and social-professional networks, we have laid a sound platform to aid understanding and provide theoretical and empirical building blocks to our research studies.
Objective measures are hard to achieve in social science research, as is generalisability, thus we have incorporated triangulation techniques in our design and seek transferability of findings rather than strict generalisability. Nevertheless, our results by definition will fall short of the gold standard, and will not produce level 1 evidence. Balancing this limitation is the reality that a strict test of this kind cannot be applied against research in this mould. Thus, other safeguards, including seeking data from multiple sources and assessing the extent to which they converge or diverge, constitute an effective, alternative approach.
This is a large, comprehensive research project which will deliver outcomes both specifically relevant to health systems and transferable to the large number of organisations, systems and sectors which are affected by the effectiveness, or otherwise, of communities of practice and social-professional networks. In developing a rigorous evaluation methodology and delivering tools for researchers, policymakers, practitioners and consumers, the research project also demonstrates innovation and tests novel approaches in design. By interpreting and translating results for applications both within and beyond the health system, the project will advance understanding of how professionals work together, and how to optimise this interaction to improve organisational effectiveness and service delivery.
This research was supported under Australian Research Council's Discovery Projects funding scheme (project number DP0986493).
Centre for Clinical Governance Research, Faculty of Medicine, University of New South Wales
Australian Institute of Health Innovation, Faculty of Medicine, University of New South Wales
School of Public Health and Community Medicine, Faculty of Medicine, University of New South Wales
Health Informatics Research and Evaluation Unit, Faculty of Health Sciences, University of Sydney
Australian College of Health Service Executives, National Office, Macquarie Hospital
Effective Practice Program, National Institute of Clinical Studies, National Health & Medical Research Council
Clinical Excellence Commission
Australian Council on Healthcare Standards
Kvarnström S, Cedersund E: Discursive patterns in multiprofessional healthcare teams.Journal of Advanced Nursing 2006,53(2):244–252.View ArticlePubMed
Stout R, Salas E: The role of trainee knowledge structures in aviation team environments.International Journal of Aviation Psychology 1997,7(3):235–250.View ArticlePubMed
Coupland C, Blyton P, Bacon N: A longitudinal study of the influence of shop floor work teams on expressions of 'us' and 'them'.Human Relations 2005,58(8):1055–1081.View Article
McKee T: Building successful teams in the midst of transition.The Journal for Quality and Participation 2003,26(1):43.
Bleakley A: Broadening conceptions of learning in medical education: the message from teamworking.Medical Education 2006,40(2):150–157.View ArticlePubMed
Wright MC, Kaber DB: Effects of automation of information-processing functions on teamwork.Human Factors 2005,47(1):50–66.View ArticlePubMed
Rosenfeld LB, Richman JM, May SK: Information adequacy, job satisfaction and organizational culture in a dispersed-network organization.Journal of Applied Communication Research 2004,32(1):28–54.View Article
Ardichvili A, Page V, Wentling T: Motivation and barriers to participation in virtual knowledge-sharing communities of practice.Journal of Knowledge Management 2003,7(1):64–77.View Article
Haesli A, Boxall P: When knowledge management meets HR strategy: an exploration of personalization-retention and codification-recruitment configurations.The International Journal of Human Resource Management 2005,16(11):1955–1975.View Article
Pathirage CP, Amaratunga DG, Haigh RP: Tacit knowledge and organisational performance: construction industry perspective.Journal of Knowledge Management 2007,11(1):115–126.View Article
Weick K, Sutcliffe K: Managing the unexpected: resilient performance in an age of uncertainty2 Edition San Francisco, CA: John Wiley & Sons, Inc 2007.
Weingart SN, Page D: Implications for practice: challenges for healthcare leaders in fostering patient safety.Quality & Safety in Health Care 2004,13(suppl_2):ii52–56.View Article
Antonacopoulou E, Ferdinand J, Graca M, Easterby-Smith M: Dynamic capabilities and organizational learning: socio-political tensions in organizational renewal. London: Advanced Institute of Management Research Working Paper Series 2005.
Wenger E, McDermott R, Snyder W: Cultivating communities of practice Boston, Massachusetts: Harvard Business School Press 2002.
Newman MEJ, Watts DJ, Strogatz SH: Random graph models of social networks.Proceedings of the National Academy of Sciences of the United States of America 2002,19(99 (Suppl 1)):2566–2572.View Article
Braithwaite J, Runciman WB, Merry A: Towards safer, better healthcare: harnessing the natural properties of complex sociotechnical systems.Qual Saf Health Care 2009, 18:37–41.View ArticlePubMed
Gosling AS, Westbrook JI, Braithwaite J: Clinical team functioning and IT innovation: a study of the diffusion of a point-of-care online evidence system.Journal of the American Medical Informatics Association 2003, 10:246–253.View Article
Braithwaite J, Westbrook MT, Iedema R, Mallock NA, Forsyth R, Zhang K: A tale of two hospitals: assessing cultural landscapes and compositions.Social Science & Medicine 2005, 60:1149–1162.View Article
Mohr JJ, Batalden PB: Improving safety on the front lines: the role of clinical microsystems.Quality & Safety in Health Care 2002,11(1):45–50.View Article
Braithwaite J: Analysing social structural and cultural change in acute settings using a Giddens-Weick paradigmatic approach.Health Care Analysis 2006, 14:91–102.View ArticlePubMed
Braithwaite J: An empirical assessment of social structural and cultural change in acute settings.Health Care Analysis 2006, 14:185–193.View ArticlePubMed
Braithwaite J, Westbrook M, Travaglia J, Iedema R, Mallock N, Long D, Nugus P, Forsyth R, Jorm C, Pawsey M: Are health systems changing in support of patient safety? A multi-methods evaluation of education, attitudes and practice.International Journal of Health Care Quality Assurance 2007,20(7):585–601.View ArticlePubMed
Øvretveit J: Action evaluation of health programmes and changes: a handbook for a user-focused approach Oxford: Radcliffe Medical Press 2002.
Westbrook M, Braithwaite J, Travaglia J, Long D, Jorm C, Iedema R: Promoting safety: varied reactions of doctors, nurses and allied health professionals to a safety improvement program.International Journal of Health Care Quality Assurance 2007,20(7):555–571.View ArticlePubMed
Westbrook JI, Braithwaite J, Georgiou A, Ampt A, Creswick N, Coiera E, Iedema R: Multi-method evaluation of information and communication technologies in health in the context of wicked problems and socio-technical theory.Journal of the American Medical Informatics Association 2007, 14:746–755.View ArticlePubMed
Verburg RM, Andriessen JHE: The assessment of communities of practice.Knowledge and Process Management 2006,13(1):13–25.View Article
Faragher EB, Cass M, Cooper CL: The relationship between job satisfaction and health: a meta-analysis.Occupational and Environmental Medicine 2005, 62:105–112.View ArticlePubMed
Denier E: Guidelines for national indicators of subjective well-being and ill-being.Applied Research in Quality of Life 2006, 1:151–157.View Article
Holton JA: Rehumanising knowledge work through fluctuating support networks: a grounded theory.The 37th World Congress of the International Institute of Sociology: 2005; Stockholm, Sweden International Institute of Sociology 2005.
Roberts J: Limits to communities of practice.Journal of Management Studies 2006,43(3):623–639.View Article
Watson J, Ovseiko P, eds: Health care systems: major themes in health and social welfare London: Routledge 2005.
Boon H, Verhoef M, O'Hara D, Findlay B: From parallel practice to integrative health care: a conceptual framework.BMC Health Services Research 2004, 4:15.View ArticlePubMed
Creswick N, Westbrook J: Examining the socialising and problem-solving networks of clinicians on a hospital ward.
Conference Proceedings of Social Science Methodology Conference of the Australian Consortium for Social and Political Research (ACSPR) 2006.
Creswick N, Westbrook J: The medication advice-seeking network of staff in an Australian Hospital renal ward.Third International Conference on Information Technology in Health Care: Socio-technical Approaches: 2007; Amsterdam IOS Press 2007, 217–232.
Creswick N, Westbrook J: Social network analysis of medication advice-seeking interactions among staff in an Australian hospital.International Journal of Medical Informatics, in press.
Braithwaite J, Hu W, Sorensen R, Patterson R, Meyerkort S, Salkeld G, Zhang K, Mallock N, Iedema R, Betbeder-Matibet L: Report: Evaluation of the Clinical Practice Improvement Training Program. Sydney: Centre for Clinical Governance Research in Health, University of New South Wales 2002.
Cook R, Rasmussen J: "Going solid": a model of systems dynamics and consequences for patient safety.Quality & Safety in Health Care 2005, 14:130–134.View Article
Lattimer V, Brailsford S, Turnbull J, Tarnaras P, Smith H, George S, Gerard K, Maslin-Prothero S: Reviewing emergency care systems I: insights from systems dynamics modelling.Emergency Medicine Journal 2004, 21:685–691.View ArticlePubMed
Sterman JD: Business dynamics: systems thinking and modeling for a complex world Boston: McGraw-Hill 2000.
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.