Research article | Open | Open Peer Review | Published:
Challenging the holy grail of hospital accreditation: A cross sectional study of inpatient satisfaction in the field of cardiology
BMC Health Services Researchvolume 10, Article number: 120 (2010)
Subjective parameters such as quality of life or patient satisfaction gain importance as outcome parameters and benchmarks in health care. In many countries hospitals are now undergoing accreditation as mandatory or voluntary measures. It is believed but unproven that accreditations positively influence quality of care and patient satisfaction. The present study aims to assess in a defined specialty (cardiology) the relationship between patient satisfaction (as measured by the recommendation rate) and accreditation status.
Consecutive patients discharged from 25 cardiology units received a validated patient satisfaction questionnaire. Data from 3,037 patients (response rate > 55%) became available for analysis. Recommendation rate was used as primary endpoint. Different control variables such as staffing level were considered.
The 15 accredited units did not differ significantly from the 10 non-accredited units regarding main hospital (i.e. staffing levels, no. of beds) and patient (age, gender) characteristics. The primary endpoint "recommendation rate of a given hospital" for accredited hospitals (65.6%, 95% Confidence Interval (CI) 63.4 - 67.8%) and hospitals without accreditation (65.8%, 95% CI 63.1 - 68.5%) was not significantly different.
Our results support the notion that - at least in the field of cardiology - successful accreditation is not linked with measurable better quality of care as perceived by the patient and reflected by the recommendation rate of a given institution. Hospital accreditation may represent a step towards quality management, but does not seem to improve overall patient satisfaction.
A key parameter that is believed to measure quality of care in a hospital setting is patient satisfaction [1–3]. Patient satisfaction is as equally important as traditional outcome parameters such as mortality or functional status. As a result health care organizations perceive patient satisfaction as an important factor that plays a pivotal role in a competitive health care market.
Patient satisfaction can be measured utilizing standardized questionnaires or interviews . While there are published data comparing patient satisfaction of different health care providers, doctors and hospital managers operating in a competitive market are constantly pressured to implement changes to improve patient satisfaction. A number of factors are believed to influence patient satisfaction including staffing levels, infrastructure, or discharge information [5–7], which are also important topics in the accreditation process.
In recent decades there is an emerging trend that accreditation or certification is a feasible measure to improve the quality of care and patient safety . The terms 'Accreditation' and 'certification' are interchangeable; 'accreditation' is used in the USA, while 'certification' is used in Germany. For the interest of this study, the term accreditation will be used. Many hospitals have their own internal quality assurance, but also aim to meet specific external standards with regard to the standardization of clinical measures and clinical pathways. In many countries external evaluation by independent assessors resulting in accreditation are measures that are believed to improve quality of care . Indeed, in many countries hospital accreditation is voluntary or a legal requirement. In the USA, for example, it is a requirement for hospitals to have accreditation to become providers in the Medicare program . In France accreditation has been compulsory since 1996 . In Germany accreditation is voluntary while quality assurance (comparison of key outcome data) is mandatory in many disciplines. Many perceive accreditation as a measure to improve quality of care or to document superiority of the quality of care, therefore many hospitals aim for accreditation to achieve a competitive advantage . The patient's perspective is being incorporated into almost every quality measure as hospitals recognize patient satisfaction as an important outcome of health service . Because of higher competition and public reporting of hospital quality in the USA and other countries, healthcare practitioners are being scrutinized and compared with external standards. In this context, accreditation is believed to be, at least, an indicator for reasonable quality (or even superior quality of care if accreditation is voluntary) that can be easily identified by patients and referring doctors. If an organization does not go through an accreditation process, it may indicate that the facility is not open to external evaluation of its performance and may lead to competitive disadvantage . Health care planners, regulators and payers are strongly supporting accreditation while physicians are more hesitant since accreditation is frequently perceived as a bureaucratic exercise that distracts from the core activities. Appropriate accreditation is an arduous process that requires substantial resources, however studies investigating its beneficial effects regarding key outcome parameters are widely lacking . As stated above, the factors on which patient satisfaction are influenced correspond widely to the topics on which accreditation processes are based on. Patient satisfaction is often a main topic in the accreditation process, as patient satisfaction is dependent on the perceived quality of information, communication and organization . These topics are also important to succeed in the accreditation process .
A number of previous studies have investigated the benefits of accreditation of health care providers , and describe the different accreditation standards. Many papers have addressed accreditation standards which are similar to American or Australian standardization. In addition, some researchers indicate that the accreditation improves their operations and performance in terms of effectiveness and efficiency . All accreditation efforts require resources and regarding the aim of evidence based management the rationale for allocation of resources and the return of investment should be measured. Interestingly, data quantifying the economical and quality effect of accreditation are widely lacking. As emphasised by Greenfield & Braithwaite  there are limited data on the influence of accreditation of health care providers on patient satisfaction. The existing studies [17, 18] have major limitations, i.e. small sample sizes or not administering validated instruments to assess patient satisfaction. However, these studies did not find an association between patient satisfaction and hospital accreditation. Other studies looking at performance measures like Patient Safety Indicators or survival rates on special diseases and accreditation did not find any or no clear relationships between them as well [19, 20]. Whether accrediting of hospitals is truly ensuring high quality healthcare is a question that remains to be answered, which indicates there is the need to provide evidence that accreditation procedures indeed result in improved patient satisfaction.
The present study aims to assess in a defined speciality (cardiology) the relationship between patient satisfaction (as measured by the recommendation rate) and accreditation status. We hypothesized that accredited hospitals have higher patient satisfaction when compared with non-accredited hospitals.
A widely accepted accreditation system designed for hospitals in Germany is KTQ (Cooperation for Transparency and Quality in Hospitals) . An alternative model is proCum Cert (pCC). Both models are very similar to the JCAHO (Joint Commission on Accreditation of Healthcare Organizations) accreditation standards which mean that a hospital undergoes an independent review of the organization's performance against national quality and safety requirements. They aim towards an organization wide accreditation. The KTQ- and pCC-accreditation processes consist of a self assessment followed by an external assessment. Both procedures are based on a nearly identical criteria-catalogue .
In 2007 a regional patient survey was conducted involving 75 hospitals in the Ruhr area, comprising of more than 50% of all hospitals in a highly urbanized area with greater than six million inhabitants. The Picker questionnaire was mailed to 78,508 patients [21, 22] between two and eight weeks after their discharge. Patients who did not answer the questionnaire were sent a first, and if necessary a second reminder four weeks after the last letter. A total of 44,418 (57%) responded. For our analysis we focused on patients discharged from dedicated departments of cardiology in order to decrease variability due to a large number of different diagnoses and procedures. These departments could be identified because the discharging unit was mentioned on the Picker questionnaire. In our study, accreditation was defined as the hospitals having received approval and received a certificate. Nine out of 25 cardiology departments were certified according to KTQ, six according to both KTQ and pCC. The remaining ten departments were not accredited. 1,835 patients were treated in an accredited unit, whereas 1,202 patients were treated in non-accredited units. Two departments were excluded because of an accreditation system that is not comparable to KTQ and pCC. We included patients treated in one of 25 specialized cardiology departments leaving a sample size of 3,037 patients.
The Picker Inpatient Questionnaire assesses seven dimensions of patient satisfaction . Jenkinson et al. demonstrated in a study over five countries that the problem scores for many of the Picker-Items are often high (>20%), but the study also indicated that the rates are not comparable between different hospitals and countries . However, a parameter that reflects overall satisfaction is the 'recommendation rate' of a specific health care provider . This is the proportion of patients that would recommend a specific institution based upon their experiences. This parameter was used as primary outcome parameter.
The following study is based on a Picker Questionnaire and Picker factors that were developed for the special needs of hospitals in Germany. The German factors differ from the Picker dimensions in Jenkinson et al. .
All individual questions that measured potential problems were dichotomized, that is, i.e. were assigned to one of two groups depending on whether a problem was mentioned . A problem score for each of the ten Picker factors was calculated on the basis of this problem rating. The problem score reflects the percentage of responses to questions that indicated a problem with care. Higher scores indicate more problems. The mean problem scores for factors of care were compared between the hospitals.
Mean values and STD or percentages with 95% confidence intervals (CI) are reported where appropriate. All p values calculated were two-tailed; non parametric tests were used where appropriate. A p-value < 0.05 was considered significant for the primary parameters with Bonferroni's correction to control for type I error for the secondary outcome parameters. Different control variables, such as staffing level were analysed between the two groups (accredited vs. non-accredited) (tables 1 and 2). Mann Whitney U test was used to compare descriptive statistics for patients in accredited and non-accredited hospitals (table 1) and to compare hospital characteristics (table 2). Mean problem scores by factors of the Picker questionnaire for accredited and non-accredited hospitals were compared using Mann Whitney U test (table 3). Patients' overall assessment of the quality of care for accredited and non-accredited hospitals was analysed using Chi Square test and Mann Whitney U test (table 4). Figure 1 shows the recommendation rate for every hospital subdivided in hospitals with and without academic affiliation and in hospitals with and without accreditation. Using Chi Square test we compared the mean recommendation rates for these groups.
As the survey was anonymous and the questionnaires were sent to the Picker Institute and not to the hospital, an ethical approval was not necessary. All statistical analyses were made with the Statistical Package for the Social Sciences (SPSS Inc., Chicago, Illinois, USA, Release 13.0, 2004).
The key characteristics of accredited and non-accredited hospitals are depicted in tables 1-2. There were no significant differences between accredited and non-accredited hospitals for length of stay, gender, age or the proportion of foreign patients. However, there was a trend that accredited hospitals were more likely to be non-profit and not affiliated teaching hospitals (p < 0.15).
Rates of institutions with accreditations (65.6%, 95% CI 63.4 - 67.8%) were not significantly different from institutions without formal accreditation (65.8%, 95% CI 63.1 - 68.5%, Figure 1).
Specific factors of patient satisfaction
With respect to the factor specific problem scores, key factors of patient satisfaction indicated no significant advantage for accredited units (table 3). We found no significant difference between the two hospital groups regarding the physician and nursing staff as perceived by patients or regarding satisfaction with the admission or discharge procedures. Overall quality ratings for cooperation of medical and nursing staff as perceived by patients, overall evaluation of care and assessment of duration of stay were not significantly different (table 4) between the two hospital groups.
The main finding of this cross sectional study which included more than 3,000 patients discharged from cardiology units from 25 different hospitals after inpatient treatment was that there is no significant difference for the recommendation rate between accredited (65.6%, 95% CI 63.4 - 67.8%) and non-accredited (65.8%, 95% CI 63.1 - 68.5%) hospitals. While it might be argued that any advantage of accredited hospitals might be simply missed due to a type II error, it is important to note that the recommendation rates of some hospitals were numerically higher in non-accredited units. In addition, more than 3,000 patients were included into this study and we made a purposeful selection of patients by specialty, therefore it is highly unlikely that the somehow surprising result is due to insufficient sample size.
A recommendation rate above 60% is considered high, however, there is still substantial room for improvement. Thus a ceiling effect that probably makes it difficult to identify differences can be excluded as a potential cause for the lack of a difference. In addition, we assessed numerous other secondary outcome parameters such as perceived quality of admission or discharge procedures. Similar to the primary outcome parameter, there were no significant differences. The only exception was the quality rating for the room atmosphere with a trend in favour of non-accredited institutions.
Accreditation procedures are believed to be suitable measures to improve quality of care including patient satisfaction. But our results indicate that hospital accreditation process does not correlate with patient satisfaction and service quality as perceived by the patient, therefore providing evidence that an accreditation is not linked to improved patient satisfaction.
As stated above, there are many studies which have investigated the benefits of accreditation of health care providers [16, 25–27]. Accreditation may be advantageous regarding standardization of procedures, cost containment or marketing. However, there are reservations that accreditation is a suitable instrument for quality improvements that are relevant to patient satisfaction. Hospitals are accredited for their compliance with standards [28, 29]. Indeed, accreditation programs focus primarily on structures and processes in patient care, e.g. the patients and their way from admission to discharge . The underlying assumption is that if hospital pathways and processes are properly regulated and controlled, patient outcomes and patient satisfaction are likely to be improved. To increase outcome orientation in the accreditation process, there is a need for quality indicators such as morbidity adjusted mortality and patient satisfaction. But until now, there are only few examples of performance measures in the accreditation process standards .
While cost containment in hospitals is an issue in many countries, there is an obvious need to correctly establish costs and benefits of accreditation. The process of accreditation requires resources and time [16, 31]. For example, Staines  describes the process of a ISO accreditation of a small Swiss hospital, required three years of work from three full time staff members. This figure may reflect only a small proportion of the true costs of an accreditation process, as implementation also requires substantial input from a large number of other stakeholders, such as doctors and nursing staff. For an accreditation program to be cost effective the gain needs to be measurable and improvement of patient satisfaction is an important outcome parameter. Considering this, it is surprising that until now there is limited data that quantify the influence of accreditation on patient satisfaction.
It is important to note that our study also has some limitations which need to be considered but are unlikely to explain the lack of an association between patient satisfaction and accreditation. Firstly, accreditation is based on a continuous quality improvement process. For this reason, it is hard to define the endpoints of accreditation so that changes in process and outcome measures may develop over time. In addition, many hospitals in our study had just received their certificate, thus they have taken significant steps within the accreditation process, but the full benefits may emerge at a later time point. We do not know exactly when the hospitals received their certificates, but we assume that it was recently as accreditation of hospitals was only introduced to Germany in 2008. This may explain why, in our study, some non-accredited institutions were numerically better for recommendation rates, and it is unlikely that the time effect will change the results. Secondly, it is possible that some hospitals were given provisional accreditation with recommendations for improvement. If so, hospitals may have been accredited despite their existing problems. Thirdly, individual accreditation programs vary with respect to scope and standards and we did not consider these differences between the two accreditation programs. However, considering the large sample size it is unlikely that any relevant association would have been missed. In addition, it could be argued that hospitals with accreditation would be worse if they did not undergo the accreditation process. However, there is no evidence (e.g. based upon the change of case load in the past) to support this assumption.
In summary, this study including more than 3,000 patients discharged from 25 cardiology units after inpatient treatment does not demonstrate any significant association between hospital accreditation and the primary or secondary outcome parameters reflecting patient satisfaction (recommendation rate and satisfaction with care). Hospital managers frequently see accreditation as a means to maintain high standards in health care delivery. However, our data may suggest that at least some of the currently established accreditations fail to improve patient satisfaction. This can be considered an important issue and further research is needed to identify the factors that determine patient satisfaction that can be effectively targeted. In addition, our data may lead towards the need to properly assess the effects of accreditation to provide data that are needed for evidence based decision making when it comes to future improvements of accreditation in health care.
Perhaps, the focus of our study on cardiology units leaves the question if results could be replicated or if they would differ if the study was conducted in patients within another medical discipline. This question is an important subject for future research.
Cleary PD, et al: Using patient reports to improve medical care: a preliminary report from 10 hospitals. Qual Manag Health Care. 1993, 2 (1): 31-8. 10.1136/qshc.2.1.31.
Guzman PM, et al: Tapping patient satisfaction: a strategy for quality assessment. Patient Educ Couns. 1988, 12 (3): 225-33. 10.1016/0738-3991(88)90006-7.
Nelson-Wernick E, et al: Patient perception of medical care. Health Care Manage Rev. 1981, 6 (1): 65-72.
Hyrkas K, Paunonen M: Patient satisfaction and research-related problems (Part 2). Is triangulation the answer?. J Nurs Manag. 2000, 8 (4): 237-45. 10.1046/j.1365-2834.2000.00176.x.
Hall JA, Dornan MC: Patient sociodemographic characteristics as predictors of satisfaction with medical care: a meta-analysis. Soc Sci Med. 1990, 30 (7): 811-8. 10.1016/0277-9536(90)90205-7.
Sjetne IS, Veenstra M, Stavem K: The effect of hospital size and teaching status on patient experiences with hospital care: a multilevel analysis. Med Care. 2007, 45 (3): 252-8. 10.1097/01.mlr.0000252162.78915.62.
McGillis Hall L, et al: Nurse staffing models as predictors of patient outcomes. Med Care. 2003, 41 (9): 1096-109. 10.1097/01.MLR.0000084180.07121.2B.
Shaw CD, Collins CD: Health service accreditation: report of a pilot programme for community hospitals. Bmj. 1995, 310 (6982): 781-4.
Shaw CD: External quality mechanisms for health care: summary of the ExPeRT project on visitatie, accreditation, EFQM and ISO assessment in European Union countries. External Peer Review Techniques. European Foundation for Quality Management. International Organization for Standardization. Int J Qual Health Care. 2000, 12 (3): 169-75. 10.1093/intqhc/12.3.169.
Sprague L: Hospital oversight in Medicare: accreditation and deeming authority. NHPF Issue Brief. 2005, 1-15. 802
de Pouvourville G: Quality of care initiatives in the French context. Int J Qual Health Care. 1997, 9 (3): 163-70. 10.1093/intqhc/9.3.163-a.
Glück D, Selbmann H-K: Genuin-medizinische Zertifizierungsverfahren in ausländischen Krankenhäusern. Z ärztl Fortbild Qual sich (ZaeFQ). 2000, 94: 659-664.
Klotz T: The determination of patient satisfaction as a part of quality management in the hospital. Dtsch Med Wochenschr. 1996, 121 (28-29): 889-95. 10.1055/s-2008-1043083.
Selbmann HK: Assessment and certification of hospital care in Germany. Bundesgesundheitsblatt Gesundheitsforschung Gesundheitsschutz. 2004, 47 (2): 103-10. 10.1007/s00103-003-0769-0.
Greenfield D, Braithwaite J: Health sector accreditation research: a systematic review. Int J Qual Health Care. 2008, 20 (3): 172-83. 10.1093/intqhc/mzn005.
Helbig M, et al: Quality assessment according to DIN EN ISO 9001:2000: certification in a university ENT department. HNO. 2006, 54 (12): 922-8. 10.1007/s00106-006-1467-7.
Heuer AJ: Hospital accreditation and patient satisfaction: testing the relationship. J Healthc Qual. 2004, 26 (1): 46-51.
Fong J, et al: Hospital quality performance report: an application of composite scoring. Am J Med Qual. 2008, 23 (4): 287-95. 10.1177/1062860608317064.
Chen J, et al: JCAHO accreditation and quality of care for acute myocardial infarction. Health Aff (Millwood). 2003, 22 (2): 243-54. 10.1377/hlthaff.22.2.243.
Miller MR, et al: Relationship between performance measurement and accreditation: implications for quality of care and patient safety. Am J Med Qual. 2005, 20 (5): 239-52. 10.1177/1062860605277076.
Bruster S, et al: National survey of hospital patients. British Medical Journal. 1994, 309: 1542-1549.
Cleary PD, et al: Patients evaluate their hospital care: a national survey. Health Aff (Millwood). 1991, 10 (4): 254-67. 10.1377/hlthaff.10.4.254.
Jenkinson C, Coulter A, Bruster S: The Picker Patient Experience Questionnaire: development and validation using data from in-patient surveys in five countries. Int J Qual Health Care. 2002, 14 (5): 353-8. 10.1093/intqhc/14.5.353.
Jenkinson C, et al: Patients' experiences and satisfaction with health care: results of a questionnaire study of specific aspects of care. Qual Saf Health Care. 2002, 11 (4): 335-9. 10.1136/qhc.11.4.335.
Casey MaJK: HMOs serving rural areas: experiences with HMO accreditation and HEDIS reporting. Manag Care Q. 2000, 8 (2): 48-59.
Schyve PM: The evolution of external quality evaluation: observations from the Joint Commission on Accreditation of Healthcare Organizations. Int J Qual Health Care. 2000, 12 (3): 255-8. 10.1093/intqhc/12.3.255.
Beholz S, Koch C, Konertz W: Quality management system of a university cardiac surgery department according to DIN EN ISO 9001: 2000. Thorac Cardiovasc Surg. 2003, 51 (3): 167-73. 10.1055/s-2003-40312.
Weiler T, Hoffmann R, Strehlau-Schwoll H: Quality management and certification. Optimizing hospital procedures. Unfallchirurg. 2003, 106 (8): 692-7. 10.1007/s00113-003-0635-4.
McAlary B: Janforum: the reliability and validity of hospital accreditation in Australia. J Adv Nurs. 1981, 6 (5): 409-11. 10.1111/j.1365-2648.1981.tb03242.x.
Collopy BT: Clinical indicators in accreditation: an effective stimulus to improve patient care. Int J Qual Health Care. 2000, 12 (3): 211-6. 10.1093/intqhc/12.3.211.
Staines A: Benefits of an ISO 9001 certification--the case of a Swiss regional hospital. Int J Health Care Qual Assur Inc Leadersh Health Serv. 2000, 13 (1): 27-33.
The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6963/10/120/prepub
The authors declare that they have no competing interests.
CS contributed to the drafting of the paper, performed statistical analysis, analysed and interpreted data and participated in the critical revision of the manuscript for important intellectual content. PL analysed and interpreted data and participated in the critical revision of the manuscript for important intellectual content. WG participated in the data acquisition and in the critical revision of the manuscript for important intellectual content. RE participated in the conception and design and in the critical revision of the manuscript for important intellectual content. KJ participated in the conception and design and in the critical revision of the manuscript for important intellectual content. GH had the idea for this study contributed to paper drafting, performed statistical analysis, analysed and interpreted data and participated in the critical revision of the manuscript for important intellectual content. All authors read and approved the final manuscript.