Measuring safety culture in Dutch primary care: psychometric characteristics of the SCOPE-PC questionnaire
© Verbakel et al.; licensee BioMed Central Ltd. 2013
Received: 19 July 2012
Accepted: 12 September 2013
Published: 17 September 2013
Patient safety has been a priority in primary healthcare in the last years. The prevailing culture is seen as an important condition for patient safety in practice and several tools to measure patient safety culture have therefore been developed. Although Dutch primary care consists of different professions, such as general practice, dental care, dietetics, physiotherapy and midwifery, a safety culture questionnaire was only available for general practices. The purpose of this study was to modify and validate this existing questionnaire to a generic questionnaire for all professions in Dutch primary care.
A validated Dutch questionnaire for general practices was modified to make it usable for all Dutch primary care professions. Subsequently, this questionnaire was administered to a random sample of 2400 practices from eleven primary care professions. The instrument’s factor structure, reliability and validity were examined using confirmatory and explorative factor analyses.
921 questionnaires were returned. Of these, 615 were eligible for factor analysis. The resulting SCOPE-PC questionnaire consisted of seven dimensions: ‘open communication and learning from errors’, ‘handover and teamwork’, ‘adequate procedures and working conditions’, ‘patient safety management’, ‘support and fellowship’, ‘intention to report events’ and ‘organisational learning’ with a total of 41 items. All dimensions had good reliability with Cronbach’s alphas ranging from 0.70 – 0.90, and the questionnaire had a good construct validity.
The SCOPE-PC questionnaire has sound psychometric characteristics for use by the different professions in Dutch primary care to gain insight in their safety culture.
KeywordsPrimary care Patient safety Safety culture Questionnaire Psychometric characteristics
One of the main focuses in patient safety research is patient safety culture. A supportive patient safety culture is seen as an important condition for patient safety . Patient safety culture refers to values, attitudes, norms, beliefs, practices, policies, and behaviours about safety issues in daily practice. In essence, culture is “the way we do things around here” . In a review, Sammer et al. identified seven subcultures of patient safety culture: leadership, teamwork, evidence based, communication, learning, just, and patient-centred . Gaining insight in the prevailing safety culture is therefore seen as a first pivotal step towards an adequate patient safety system . Various instruments have been developed to measure patient safety culture [2, 5–8]. They help to identify weak areas in the perceived safety culture and thus enable designing tailored improvement strategies.
In recent years, increasing attention has been given to patient safety in primary care [9–14]. Primary care is directly accessible and consists of a broad array of professions, e.g. dental care, general practice, physiotherapy, midwifery, speech therapy. Despite this wide range of care, practices have many similarities in organisational structure. As most practices are small, managerial and organisational tasks -including safety improvement- are mostly done by the professionals themselves. Moreover, primary care professionals increasingly work together in broad healthcare centres, collaborating in disease management programmes and consulting one another in managing the care of individual patients.
Because of the increase in collaboration within primary care, developing a generic patient safety culture instrument was desirable. It will enable comparison between different primary care providers and in a later stage of safety management, may generate exchange of learning and improvement strategies. As a tool for patient safety culture already exists in the Netherlands: the SCOPE, it has been developed and validated for general practice only . SCOPE is a Dutch acronym for systematic culture inquiry on patient safety. Other primary care professionals were already familiar with it, therefore, we choose to modify this tool into a generic questionnaire for all professions in primary care: the SCOPE-Primary Care (SCOPE-PC).
First adjustment to the questionnaire
Handover and teamwork (8 items)
Support and fellowship (5 items)
Communication openness (6 items)
Feedback about and learning from error (6 items)
Intention to report events (3 items)
Adequate procedures and adequate staffing (7 items)
Overall perceptions of patient safety management (4 items)
Expectations and actions of managers (4 items).
We made adjustments to this SCOPE through an iterative process. First, the research team revised the terminology of the questionnaire. Secondly, professionals from all primary care professions assessed the questionnaire individually clarity and applicability to their own setting. In total 27 professionals (1 midwife, 4 pharmacists, 1 physician, 2 dieticians, 2 physician assistants, 2 physiotherapists, 2 skin therapists, 2 general practitioners, 1 speech therapist, 2 dental hygienists, 2 exercise therapists, 2, dentists, 2 dentist assistants, 1 occupational therapist, 1 general practice nurse and 1 nurse working in an anticoagulation clinic) gave feedback by e-mail. Lastly, the research team reached consensus on the version to be used for the further validation process.
Adjustments were limited to a few changes of terminology, for example ‘general practitioner’ was changed to ‘professional’ and ‘physician assistant’ was changed to ‘support staff’. None of the original patient safety culture items were deleted. Three questions were added for routing purposes, where if this question prompted a negative answer the respondent was not shown the other questions regarding this topic. One question was added: ‘Are incident reports discussed in meetings on a structural basis? Structural means that it is a permanent feature on the agenda’.
Besides patient safety culture questions the existing questions about patient safety characteristics of the practice were included: whether or not events were discussed in an informal way, the frequency of event reports filled in in the last 12 months and a patient safety grade for the total practice (answer categories: failing, poor, acceptable, good, excellent).
The final questionnaire consisted of 43 patient safety culture items. Items had to be answered using a five-point Likert scale ranging from strongly disagree (1) to strongly agree (5) or never (1) to always (5). On request of the individual professionals we added the option ‘not applicable’ to the questions about the practice organisation and collaboration. Background questions addressed demographics and work-related information, such as how long and in which profession the respondent had been working in this practice.
Data collection and respondents
Data collection for validation of the questionnaire took place from March until May 2011. An online system managed by the Dutch Practices Accreditation Organisation was used for collection and storage of the data .
Eleven primary care professions participated: dental care, dental hygienist care, dietetics, exercise therapy, physiotherapy, occupational therapy, midwifery, anticoagulation clinics, general practice, skin therapy and speech therapy. A random sample of 200 members was drawn from the national databases of each professional association. These members were asked to participate and to invite colleagues from their own practice too. The key to sign in to the digital questionnaire was included in the invitation. It was emphasized that the questionnaire was to be filled out individually. In addition, practices were promised a feedback report regarding the patient safety culture of their practice.
The selection process differed for one of the professions: the physiotherapists were invited directly by their professional association. Because of this extra step, a lower response rate was expected. To anticipate on this, the sample for physiotherapists was doubled to 400. Once enrolled, the inclusion and the following steps were the same as for the other professions.
All practices received a first invitation followed by two reminders with an interval of three weeks to all the contact persons. Invitations and reminders were preferably sent by e-mail but if not available by post.
As culture is a feature of a group, single-handed practices without employees were excluded from analyses. In addition, as it takes time to absorb the culture of an organisation, we excluded respondents with less than half a year experience in their current practice. Further, respondents with more than five missing values on the patient safety culture items were excluded. The answer category ‘not applicable’ was not counted as missing. Items that were negatively worded were recoded so that high scores always reflect a positive response. Subsequently, distributions of variables were examined to assess response variability and missing data. Inter-item correlations were studied, as well as Bartlett’s test of sphericity and the Kaiser-Meyer-Olkin Measure of Sampling Adequacy (KMO) were performed to see whether a factor analysis could be performed. When Bartlett’s test is significant (p<0.001) it indicates that the data are appropriate for factor analysis. For KMO a value near 1 indicates that patterns of correlations are relatively compact and factor analysis should yield distinct and reliable factors . Regarding the rule of thumb of 10 respondents per patient safety culture item, at least 430 completed questionnaires were needed .
As we built on an existing questionnaire, a confirmatory factor analysis (CFA) was performed to investigate whether the structure of the original SCOPE for general practices could be confirmed for these data (for explanation of the CFA see Additional file 1). The χ2 and RMSEA were used as parameters for goodness of fit. A non-significant χ2 means that the discrepancies between the hypothesized model and the empirical data are negligibly small and thus indicate a good fit. The RMSEA measures how well the empirical model approaches the theoretical model. A value of <0.05 is considered a close fit of the model, a value of <0.08 fair or a reasonable error of approximation, and values >0.1 are regarded as not acceptable [20, 21].
To examine whether a different structure would give a better fit to the data, an explorative factor analysis (principal component analysis, promax rotation) was performed. To determine how many factors should be retained the eigenvalues and the scree plot (see Additional file 2) were examined. Also, the total explained variance was taken into account.
Internal consistency of the factors was measured using Cronbach’s alpha. A Cronbach’s alpha of >.60 indicates that different items measure the same concept . A positive rating for internal consistency is met when Cronbach’s alphas range between .70 and .95 . We also examined the deleted-item reliability coefficients.
For all respondents, sum scores were calculated by obtaining the mean score of all items within one dimension. One missing value per dimension was allowed. Subsequently, intercorrelations between dimensions were calculated with Pearson correlation coefficients. We expected that the various dimensions would correlate moderately as they cover an aspect of the same construct: patient safety culture. However, the correlations should not exceed .70 because this would mean that the dimensions are too similar and measure the same concept. Furthermore, correlations of the dimensions with the patient safety grade were computed. It was expected that all dimensions would have a positive correlation with the grade.
All Statistical analyses were conducted using SPSS 17.0 and Lisrel 8.8 for the CFA .
The Medical Research Ethics Committee of the University Medical Center Utrecht concluded that no WMO approval for this study was needed.
In total, 921 individual questionnaires were returned from 519 practices. 306 questionnaires were excluded for further analysis: 200 from single-handed practices, 11 from respondents with less than half a year experience at the particular practice and 94 with more than 5 missing values, resulting in 615 questionnaires eligible for the study. Bartlett’s test was significant (p<0.001) and the KMO was 0.91 indicating that the data were appropriate for a factor analysis.
Study population by age and gender and response rate per profession
Age mean (SD)
Exercise therapy practices
Dental care practices
Dental hygiene practices
Occupational therapy practices
Skin therapy practices
Speech therapy practices
Confirmatory factor analysis
A confirmatory factor analysis showed that the original factor structure did not fit well with the data (χ2=2855.56 df=832, p<0.001). The RMSEA was .06 (90% confidence interval: .06-.07). Although some authors consider a value of .06 a fair fit, the cut-off point of <.05 is usually used. Therefore, also an exploratory factor analysis was conducted to examine if a different structure would fit the data better.
Exploratory factor analysis
Mean scores and factor loadings of the items of the SCOPE-PC questionnaire
Open communication and learning from error
We are given feedback about changes put into place based on event reports
Staff will freely speak up if they see something that may negatively affect patient care
We are informed about errors that happen in this practice
Staff feel free to question the decisions or actions of those with more authority
In this practice, we discuss ways to prevent errors from happening again
Professionals discuss errors that occurred with each other
We are given personal feedback about our own event reports
My supervisor/manager overlooks patient safety problems that happen over and over
Handover and teamwork
Problems often occur in the exchange of information across disciplines in our practice
The fact that patients are treated by different professionals in our practice is causing problems
Disciplines in the practice that we co work with do not coordinate well with each other
There is a good exchange of information between professionals in this practice
There is a good exchange of information between supporting staff in this practice
Things “fall between the cracks” when transferring patients between different disciplines in this practice.
Important patient care information is often lost because patients see different professionals
Adequate procedures and working conditions
It is just by chance that more serious mistakes don’t happen around here
We use more agency/temporary staff than is best for patient care
Staff feel like their mistakes are held against them
In this practice we work longer hours than is best for patient care
When an event is reported, it feels like the person is being written up, not the problem
We work in “crisis mode” trying to do too much, too quickly
Staff worry that mistakes they make are kept in their personnel file
We have patient safety problems in this practice
Whenever pressure builds up, my supervisor/manager wants us to work faster, even if it means taking shortcuts
Patient safety management
My supervisor/manager says a good word when he/she sees a job done according to established patient safety procedures
My supervisor/manager seriously considers staff suggestions for improving patient safety
My supervisor/manager provides a work climate that promotes patient safety
The actions of my supervisor/manager show that patient safety is top priority
My supervisor/manager seems interested in patient safety only after an adverse event happens
Support and fellowship
People support one another in this practice
We have enough staff to handle the workload
When a lot of work needs to be done quickly, we work together as a team to get the work done
In this practice, people treat each other with respect
When someone in this practice gets really busy, others help out
Intention to report events
When a mistake is made, but is caught and corrected before affecting the patient, how often is this reported?
When a mistake is made, but has no potential to harm the patient, how often is this reported?
When a mistake is made that could harm the patient, but does not, how often is this reported?
We are actively doing things to improve patient safety
Mistakes have led to positive changes here
Our procedures and systems are good at preventing errors from happening
Staff are afraid to ask questions when something does not seem right
Disciplines work together well to provide the best care for patients
Professionals discuss errors that occurred with other disciplines
Two items did not have a satisfactory loading above .40 on any of the factors and one item loaded on two factors. Two of these items: ‘Staff are afraid to raise questions if something does not seem right’ and ‘Disciplines work together well to provide the best care for patients’ were deleted from the questionnaire, since the content of these items were covered to a great extent by other items. As the content of the item ‘Professionals discuss errors that occurred with other disciplines’ was not covered by other items, and in view of the expected increase of interdisciplinary collaboration in the near future, it was retained as a separate item. A few items fell under a different dimensions than in the original SCOPE. The answer category ‘not applicable’ was retained for the question ‘When one area in this practice gets really busy, others help out’ and questions regarding collaboration. The seven dimensions jointly explained 58.9% of variance.
Reliability and construct validity
Mean dimension scores, correlation with patient safety grade and intercorrelations of the seven dimensions
Patient safety grade
Open communication and learning from error
Handover and teamwork
Adequate procedures and working conditions
Patient safety management
Support and fellowship
Intention to report events
Validation of the SCOPE-PC showed that the questionnaire consisted of seven dimensions, slightly differing from the original SCOPE questionnaire with eight dimensions. The main difference was that the original dimension ‘communication openness’ in the current study was divided in two dimensions: ‘open communication and learning from error’, and ‘adequate procedures and working conditions’. Internal consistency and construct validity were good.
Interpretation of findings
It is interesting to note the absence of a correlation between ‘intention to report events’ and all other dimensions but one: ‘open communication and learning from error’. The absence of correlation between ‘intention to report’ and most other dimensions may be explained by a difference in perspective. The questions about reporting relate to actual steps to be undertaken when an error occurs, they ask about one’s personal intentions: What would you do if? In contrast, questions regarding collaboration, support, the notion of abiding and employing the procedures about patient safety relate to how everybody feels or thinks of the atmosphere in their practice, and is concerned with how this is at the moment.
Another explanation for the absence of correlation could be the fact that reporting is still very uncommon in primary care. The dimension ‘intention to report events’ does therefore not ‘behave’ the way the other dimensions do. Additionally, the fact that ‘open communication and learning from error’ does correlate may indicate that this is an important precondition for reporting. Subsequently, one would expect that the coherence of all dimensions will become stronger as safety management activities become common practice.
The uneven distribution of the response rate in the eleven professions was striking. A partial explanation is that some professionals, like general practitioners, often receive requests to participate in a study and are therefore less likely to respond. Also, some general practitioners had already completed the original questionnaire during their accreditation process. The original questionnaire was already available for them. Furthermore, it is possible that the interest for patient safety differs between primary care professions. Despite these differences we considered all professions as one group; similarities in the targeted patient group, organisational structure and educational level of employees justify this.
Strengths and limitations
The strength of this questionnaire is that it serves all primary care professions with one generic questionnaire. This facilitates comparison in further research.
This study has some limitations. First, selection bias due to the fact that more innovative practices and practices that are more enthusiastic about the topic were probably more willing to participate, could not be excluded. Because data collection was done through a contact person, we may assume that personal drive and maybe even authority played a role in participation of individuals. However, for a psychometric study this is less important since the focus is on clustering of items.
Second, the response rate of 38.4% for individual questionnaires is not very high, yet it is not unusual for an open population study. The low response could be due to the fact that because of the organisational structure, primary care professionals have no overhead time for such activities. On top of this, data collection was hindered because the membership records of some groups were not up to date. In addition, we were not able to distinguish the single-handed practices beforehand. Still, with a total of 615 questionnaires the rule of thumb of a ratio of ten respondents per item is amply met with a ratio of 14:1.
Third, a general drawback of measuring culture with a survey is that it will not capture the heart of the current culture, for which a more sophisticated method is necessary . Options given are participant observation, interviews and focus groups combined with attitudinal surveys and established cultural assessment tools. Indeed, it would be interesting to combine the SCOPE-PC questionnaire with qualitative methods in a future study aiming at describing patient safety culture. However, for professionals themselves, as final users of the product, a survey has the advantage that it is feasible and easy to use.
In conclusion, this study showed that the SCOPE-PC questionnaire for primary care has sound psychometric properties. The questionnaire is slightly different from the original SCOPE, but overall the main part of the factor structure is the same and only two items were deleted. In future, when sufficient data will be available, it would be interesting to perform cross-validation of the questionnaire. The use of the questionnaire will enable all professions in primary care to gain insight in their safety culture status and to take steps from there to improve patient safety in their practices. In our opinion, the next step in research is to explore the status and possible differences between professions in the Dutch primary care regarding patient safety culture.
We would like to thank the primary care professionals that contributed to the study by completing the questionnaires, and the Dutch Practice Accreditation (NHG Praktijkaccreditering®) for their contribution to the data collection. We would also like to thank Dr. Saskia S.L. Mol for textual editing. In addition, we would like to thank the Dutch Ministry of Health for financing this project.
- Institute of Medicine: To err is human: building a safer health system. 2000, Washington D.C: National Academic PressGoogle Scholar
- Scott T, Mannion R, Davies H, Marshall M: The quantitative measurement of organisational culture in health care: a review of the available instruments. Health Serv Res. 2003, 38 (3): 923-945. 10.1111/1475-6773.00154.View ArticlePubMedPubMed CentralGoogle Scholar
- Sammer CE, Lykens K, Singh KP, Mains DA, Lackan NA: What is patient safety culture? a review of the literature. J of Nursing Scholarschip. 2010, 42 (2): 156-165. 10.1111/j.1547-5069.2009.01330.x.View ArticleGoogle Scholar
- NPSA (the National Patient Safety Agency): Seven steps to patient safety for primary care. 2006, LondonGoogle Scholar
- Nieva VF, Sorra J: Safety culture assessment: a tool for improving patient safety in healthcare organisations. Qual Saf Health Care. 2003, 12 (Suppl II): 17-23.Google Scholar
- Colla JB, Bracken AC, Kinney LM, Weeks WB: Measuring patient safety climate: a review of surveys. Qual Saf Health Care. 2005, 14: 364-366. 10.1136/qshc.2005.014217.View ArticlePubMedPubMed CentralGoogle Scholar
- Singla AK, Kitch BT, Weissman JS, Campbell EG: Assessing patient safety culture: a review and synthesis of the measurement tools. J Patient Saf. 2006, 2 (3): 105-115. 10.1097/01.jps.0000235388.39149.5a.View ArticleGoogle Scholar
- Pronovost P, Sexton B: Assessing safety culture: guidelines and recommendations. Qual Saf Health Care. 2005, 14: 231-233. 10.1136/qshc.2005.015180.View ArticlePubMedPubMed CentralGoogle Scholar
- Kirk S, Parker D, Claridge T, Esmail A, Marschall M: Patient safety culture in primary care: developing a theoretical framework for practical use. Qual Saf Health Care. 2007, 16: 313-320. 10.1136/qshc.2006.018366.View ArticlePubMedPubMed CentralGoogle Scholar
- Wallis K, Dovey S: Assessing patient safety culture in New Zealand primary care: a pilot study using a modified Manchester Patient Safety Framework in Dunedin general practices. J Prim Health Care. 2011, 3 (1): 35-40.PubMedGoogle Scholar
- Hoffmann B, Domanska OM, Albay Z, Mueller V, Guethlin C, Thomas EJ, Gerlach FM: The Frankfurt Patient Safety Climate Questionnaire for General Practices(FraSiK): analysis of psychometric properties. BMJ Qual Saf. 2011, 20: 797-805. 10.1136/bmjqs.2010.049411.View ArticlePubMedGoogle Scholar
- de Wet C, Spence W, Mash R, Johnson P, Bowie P: The development and psychometric evaluation of a safety climate measure for primary care. Qual Saf Health. 2010, 19 (6): 578-584. 10.1136/qshc.2008.031062.Google Scholar
- Gaal S, Smits M, Verstappen W, Wolters R, Lankveld H, Weel van C, Wensing M: Prevalence and consequences of patient safety incidents in general practice in the Netherlands: a retrospective medical record review study. Implementation Science. 2011, 6: 37-10.1186/1748-5908-6-37.View ArticlePubMedPubMed CentralGoogle Scholar
- Gaal S, Verstappen W, Wensing M: Patient safety in primary care: a survey of general practitioners in the Netherlands. BMC Health Serv Res. 2010, 10: 21-10.1186/1472-6963-10-21.View ArticlePubMedPubMed CentralGoogle Scholar
- Zwart DLM, Langelaan M, van de RC V, Kuyvenhoven MM, Kalkman CJ, Verheij TJM, Wagner C: Patient safety culture measurement in general practice. Clinimetric properties of ‘SCOPE’. BMC Fam Prac. 2011, 12: 117-10.1186/1471-2296-12-117.View ArticleGoogle Scholar
- Smits M, Christiaans-Dingelhoff I, Wagner C, Wal G, Groenewegen PP: The psychometric properties of the ‘Hospital Survey on Patient Safety Culture’ in Dutch hospitals. BMC Health Serv Res. 2008, 8: 230-10.1186/1472-6963-8-230.View ArticlePubMedPubMed CentralGoogle Scholar
- NHG-praktijkaccreditering® (Dutch College of General Practice-Practice Quality Accreditation). 2011
- Field A: Discovering Statistics using SPSS for Windows. 2000, London: Sage PublicationsGoogle Scholar
- Terwee CB, Bot SDM, de MR B, van der DAWM W, Knol DL, Dekker J, Bouter LM, Vet HCW: Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol. 2007, 60: 34-42. 10.1016/j.jclinepi.2006.03.012.View ArticlePubMedGoogle Scholar
- Hox JJ: Principes en toepassing van structurele modellen [Principles and application of structural models]. Kind en Adolescent. 1999, 20 (3): 200-217.Google Scholar
- Arbuckle J: Amos™ 18 User’s Guide. 2007, Crawfordville: Amos development corporationGoogle Scholar
- Jöreskog KG, Sörbom D: LISREL 8.80 for Windows [Computer Software]. 2006, Lincolnwood, IL: Scientific Software International, Inc.Google Scholar
- Marshall M, Parker D, Esmail A, Kirk S, Claridge T: Culture of Safety. Qual Saf Health Care. 2003, 12: 318-10.1136/qhc.12.4.318.View ArticlePubMedPubMed CentralGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6963/13/354/prepub