Developing 21^stcentury accreditation standards for teaching hospitals: the Taiwan experience

It has been well established that not only does hospital accreditation elevate health care quality, it also improves the abilities of health care personnel [1]. It has likewise been widely accepted that hospital accreditation is paramount to patient safety [2]. Health care is a continuous process. Therefore, we cannot only survey structures for the purpose of accreditation, but also need to evaluate outcomes and processes [3]. Accreditation standards must be predetermined, agreed upon and made public [4].

Taiwan started accrediting hospitals in 1986 when the Medical Care Act aimed at improving health care quality through regulatory reform was first enacted. Accreditation was mandatory for hospitals and initially free. At first, the Department of Health (DOH), Taiwan, conducted the accreditations by itself. After the establishment of the National Health Insurance (NHI) program in 1995, only accredited hospitals were qualified to be NHI health care providers, offering strong incentives for hospitals to seek high scores for accreditation. With time, the scope of accreditation has been expanded and elaborated, requiring 12 categories of surveyors to conduct a simultaneous on-site survey of a variety of hospital services, such as internal medicine, surgery, nursing, pharmacy, radiology, laboratory medicine, psychiatry, teaching, and administration. The total number of accredited hospitals had reached 500, and over 100 of them were teaching hospitals. The criticism the old system encountered most frequently was the standards were too structure-oriented and did not put enough emphasis on process and outcomes. Moreover, hospitals also complained that too many surveyors on site at one time had interrupted the daily operation of the hospitals. In short, the scale of accreditation grew too big for the DOH to handle with its own staff, and too burdensome for the hospitals receiving on-site visits. In 1999, the DOH delegated this duty to another organization and reform was needed in response to the health care community's critique.

The Taiwan Joint Commission on Hospital Accreditation (TJCHA) is a foundation jointly endowed by the DOH and a number of health care societies [5]. TJCHA has been charged with conducting accreditation by the DOH since 1999. Hospital accreditation is still mandatory but no longer free. From its inception, the TJCHA thus faced the challenge of revising accreditation processes and standards. It first sought to condense the 12 category approach into a four category one, i.e., medicine, nursing, management and teaching. The advantages of this approach include: first, it discourages pressure to add more specialty standards due to the demand of interest groups, and secondly, it becomes more manageable to maintain a group of qualified surveyors so as to ensure the consistency of survey results. In line with this new approach, the standards have to be rewritten and reorganized. In addition, the new standards also need to focus less on structure, instead more on process and outcome.

Teaching hospitals are defined, according to the Medical Care Act of Taiwan, as hospitals with teaching, researching and training facilities for the purposes of training physicians, paramedics, and medical and paramedical students that have passed the teaching hospital accreditation [6]. Although the number of accreditation standards with respect to teaching increased over the years, the old standards were structure-oriented and there was still no substantive standard with respect to training processes and outcomes. For instance, we have two levels of teaching hospitals, A and B. A level hospitals tend to provide tertiary care and are larger hospitals in terms of bed numbers, with over 500 acute care beds. In contrast, B level hospitals are more likely to be smaller community hospitals. Until 2004, the basic requirements for both levels in the standards included the number of qualified teaching physicians, teaching equipment, library, number of teaching activities, the hours of bedside teaching, the number of publications, and the amount of money spent on teaching, training and research, etc [7].

Although there is no identical teaching hospital accreditation system internationally to what we have in Taiwan, in the sense that we have bundled hospital accreditation and teaching hospital accreditation together, there are still other examples that we can reference in revising our standards. For instance, the World Federation for Medical Education (WFME) has established global standards for quality in basic medical education, and in postgraduate medical education. The WFME basic medical education quality improvement standards look at mission and objectives, educational program, students, student assessment, academic staff/faculty, educational resources, program evaluation, governance and administration, and continuous renewal [8]. Similarly, WFME postgraduate medical education quality improvement standards focus on mission and outcomes, training process, assessment of trainees, trainees, staffing, training settings and educational resources, evaluation of training process, governance and administration, and continuous renewal [9].

The Accreditation Council for Graduate Medical Education (ACGME) of the United States has been accrediting resident training programs for many years. ACGME has established institutional and common program requirements to regulate all resident training institutions and programs. The common program requirements in particular evaluate program personnel and resources, resident appointments, program curricula, resident duty hours and working environment, evaluation, experimentation and innovation [10]. The institutional requirements stress organization and responsibilities, responsibilities for residents, the graduate medical education committee, and internal review [11].

The purpose of this study is to establish teaching hospital accreditation standards anew with the hope that Taiwan's teaching hospitals can live up to the expectations of our society and ensure quality teaching. All in all, we hope establishing a better teaching hospital accreditation system can help cultivate excellent physicians and other health care professionals so as to elevate health care quality.

The development process lasted for two years, 2005-2006, and was separated into three stages. The first stage centered on leadership meetings and consensus building, the second on drafting the new standards with an expert focus groups, and the third on a pilot study and subsequent revision. This reform was mandated and approved by the DOH to be executed by the TJCHA, and no human subjects were involved in the process.

Eleven hospitals participated in the pilot study selected through stratified random sampling of the volunteers. There were three level A and eight level B teaching hospitals. The level A teaching hospitals are all tertiary care medical centers. Of the roughly 100 teaching hospitals, about 20 were level A teaching hospitals. Geographical distribution was taken into account; the three medical centers were in northern, central and southern Taiwan. As to the level B teaching hospitals, three are from the northern region, three from the center and two from the south.

For category two teaching and training plans and outcomes, the pass rate was 91.6% and 100% passing was achieved for 21 standards (out of 42 items). The lowest pass rate was 54.6% for standard 2.2.11: The average overall pass rate for internal medicine, surgery, obstetrics and gynecology, and pediatrics board exams for the past three years was above 80%.

For category three, research and results, the pass rate was 78.8% and 100% passing was achieved for four standards (out of nine items). This category had the lowest overall pass rate of the six categories. The lowest pass rate was 54.6% for two standards: 3.2.2 stipulating that physicians should have good research results, and 3.2.3 stating that the execution and results of paramedicals' research must be good.

For category four, development of faculty and continuing education, the pass rate was 84.1% and 100% passing was achieved for three standards (of eight items). The lowest pass rate was 63.6% for two standards: 4.1.1, requiring a faculty development system for physicians, which is effectively operational, and 4.1.4, teaching incentives for paramedicals should be clearly promulgated and executed fully so as to encourage dedication to teaching activities.

For category 5, academic exchanges and the roles and function of the hospital in the community, the pass rate was 94.4% and 100% passing was achieved for six standards (out of eight items). This category had the highest overall pass rate of the six categories. The lowest pass rate, 63.6%, was for standard 5.2.2: participating in international medical aid and disaster relief and assisting in establishing systems and personnel training.

For category 6, administration, the pass rate was 80.7% and 100% passing was achieved for only one of eight standard items. The lowest pass rate was 54.6% for standard 6.3.1, mandating separate funding for teaching, research and advanced study, which could be supported by budgeting and final accounting data, and appropriate proportionate allocation (including physicians and paramedicals).

Our current system is compatible with the trend that hospital accreditation is operated by a non-profit organization for the whole nation and linked to health care reimbursement [12]. Similar examples can be found around the globe, such as the Joint Commission [13], the Japan Council for Quality Health Care (JCQHC) [14], Accreditation Canada [15], and the Australian Council on Health care Standards (ACHS) [16]. All these systems have provided some guidance for revising our teaching accreditation standards.

Although there is no identical teaching hospital accreditation system elsewhere in the world, we still looked to the systems of Japan, Zambia, Australia, and other places for references [2, 16, 17]. Based on our own experience, in conjunction with the global trend, the new Taiwanese system focuses more on process and outcomes and has the following features.

Category 1, educational resources, includes standards for faculty, equipment, libraries, and training facilities. These are basic standards and appear to be mostly structure oriented. However, we designed them to be more functional than structural. The structural aspect, such as the amount of hardware and the credentials of faculty, can be dealt with by pre-survey file reviews so as to reduce the time surveyors need to spend verifying these items on-site, enabling them to focus more on evaluating training plans.

In this category, standard 1.1.5 had the lowest pass rate in our pilot study. This standard states that all training plans for resident doctors and medical students should have a qualified program director. As will be further elaborated below, Taiwan started medical education and post graduate training reforms after the Severe Acute Respiratory Syndrome (SARS) outbreak. The DOH has set up training programs to train the trainers, i.e. the current faculty. As such, those programs can certify hospital staff to be various qualified training program directors. The reason for low compliance rate is that it was a fairly new initiative back in 2006 and a lot of hospitals had not sent out their staff to receive relevant trainings yet. Therefore, this standard remained unchanged after the pilot since it is necessary and we believe it will improve with time.

Category 2 not only includes training for medical students, interns and resident doctors, but also for paramedicals, including students in nursing, pharmacy, radiation technology and clinical laboratory students. During on-site surveys, students and resident doctors are interviewed to evaluate the performance of training plans. The new standards stress general medicine education, not specialty training. The reasoning is that evaluating specialty training programs should fall within the jurisdiction of specialty boards, and should not be in the hands of the TJCHA.

A new post-graduate general medicine training program for physicians [18] was started in 2003 right after SARS. Taiwanese health care professionals tried to create value from this drastic experience. One of the precious lessons we learned was the need to overhaul our resident training programs to instill a greater component of general medicine into over-specialized post-graduate training. In our medical education system, medical students spend their last of seven years of medical school as interns. After graduation, they enter resident training programs of their chosen specialties right away. This highly specialized approach does not serve the needs of the community well in a time of emergency epidemic. Therefore, the new program, called the PGY1 program, requires medical graduates to participate in a three-month general medicine training program during the first year of residency. The program includes fundamental courses and training in general internal medicine, general surgery and community medicine. The goal is to expand this training program to a one-year program in 2011. Since this was a new initiative when the study started, it was not included in the old teaching hospital accreditation standards. Later, this important change was incorporated into the survey process.

The other revision worth noting is that the lowest pass rate in this category was for standard 2.2.11. The original requirement was that the average overall pass rate in internal medicine, surgery, obstetrics and gynecology, and the pediatrics board exams for the past three years should be above 80%. This turned out to be a high bar because only around 50% of the pilot-tested hospitals could get beyond it. Therefore, it was subsequently revised downward to 75%.

For category 3, we stress the importance of promulgating incentives to encourage research and reward good results. These incentives cover not only physicians but also paramedicals.

In category 4, the new standards require hospitals to emphasize faculty development and put in place incentives for teaching so as to reward excellence. Hospitals are encouraged to have bylaws that encourage attending physicians to develop teaching careers.

For category 5, we stress the importance of vertical integration among different levels of hospitals, community outreach, and international exchanges and cooperation to provide teachers and students with diverse training and broaden their perspectives.

For category 6, administration, there are four major points: the operation of the administrative system, the operation of the medical education committee, budgeting and spending, and assessing teaching and research performance.

The language of the 95 standards may appear vague in terms of giving clear guidance about how to assess compliance. We have to point out that this study is not the end of the process. We still plan to write up scoring instructions for each standard so that surveyors can assess compliance systematically. Each standard will be given four scoring levels: A, B, C and D. A is the highest grade and C is a passing grade. For instance, if we simply read the language of standard 3.2.2, which states, "Physicians should have good research results," one may be really lost as to how to assess whether results are good or bad. The scoring instructions subsequently drafted for this standard instructs surveyors to look at the percentage of full time attending physicians who have published papers in peer reviewed academic journals in the past five years. The passing grade, i.e. C, requires 25% of the full time attending physicians to publish at least one academic paper as first or correspondence author under the hospital's name. B requires 50% and A 75%. The final results of teaching accreditation hinge on the sum of all the scores. However, the relative quality and impact of the publications have not been taken into account by this approach and this shortcoming also needs to be addressed in the future.

That said, there is no denying that assessment of good and bad is always value-laden and judgmental. Although we try to create objectivity with the help of scoring instructions, we cannot escape the fact that the determination is still made through the subjective observations of the surveyors. Therefore, the objectivity and consistency of surveyors have always been a problem for which they have often been criticized in the past. The TJCHA had also noticed this issue and took measures to ensure the quality of the accreditation process itself. Starting from 2004, as part of the hospital accreditation reform, it launched a new program for recruiting and training surveyors in order to ensure performance and objectivity [19]. In the new program, surveyors are constantly trained and evaluated.

Although the final decisions as to pass and fail are supposed to be made by the DOH, the cumulative scores of the surveyors for each survey in fact dictate the final results since TJCHA has transformed the survey findings numerical by the establishment of scoring instructions. The advantage is that the chances of arbitrary manipulation by government or TJCHA officials are minimized; whereas the disadvantage is that extra mechanisms have to be put in place in order to ensure the objectivity of the scoring processes. Therefore, normalization of survey scores is needed for each cohort of hospitals in the end. The way to go about it in Taiwan is by some sort of Delphi technique; after all the hospitals for that year have been surveyed, all the surveyors will gather for a final meeting in which average scores and their variances on all standards are disclosed to all surveyors. The surveyors individually can decide whether they will adjust their scorings after knowing the distribution of scores. The adjustment made in the final meeting will be the final score for the hospitals. The TJCHA does not apply any mathematical manipulation afterwards since the assessment of each surveyor has to be respected.

Our pilot results indicate that average compliance rates with the new standards are at least around 80% for each category, which is quite high for a new set of standards. Since the pilot surveys were conducted on volunteer institutions, we need to factor in the influence of selection bias. Larger hospitals are more likely to volunteer, even among the B level teaching hospitals, and are likely to be better poised to embrace the new standards. It is therefore not surprising that the compliance rates are high. However, the new standards will be applied across the board and to smaller teaching hospitals as well. The TJCHA needs to ensure that the new standards inspire hospitals to reach new heights and yet are realistic at the same time. We cannot set a bar that is too high and doomed to fail because the TJCHA will encounter insurmountable resistance from the hospitals. Furthermore, there is 20% room for improvement, which should pose a reasonable challenge for most hospitals.

All in all, in comparison with the old standard, our new teaching hospital accreditation standards focus more on evaluating process and outcome, and are in sync with the overall trend in hospital accreditation. Research excellence appears to be the most difficult area for hospitals to achieve. It is certainly true that good teachers are not necessarily good researchers. Nonetheless, researchers are expected to be capable of enhancing teaching quality in the hospital. Although we aim at strengthening both teaching and research, our standards also look at clinical care processes in order to understand the real operation of clinical teaching and the education quality that can be achieved through the provision of hospital services. This has also been recommended to avoid false filing of accreditation data by hospitals [20].

Learning from past experience, the TJCHA spearheaded the hospital accreditation reform after its inauguration in the late 90 s. Then the SARS epidemic led to a new wave of changes in medical education and post-graduate medical training that needed to be incorporated into the teaching hospital accreditation. As such, after rigorous study and pilot testing, the new teaching hospital accreditation standards launched in the first decade of the 21^st century consist of six categories and 95 standards. Under the new standards, not only resident doctors' training but also medical students' training is evaluated. The scope of the standards have also been extended to encompass professionals in nursing, pharmacy, radiation technology and clinical laboratories. The new on-site survey focuses more on evaluating process and outcomes than the old standards. Pilot testing has proven new standards to be feasible and some pose reasonable challenges for hospitals to aspire to. Further assessment of the performance of the new set of standards is still needed once it is fully implemented.