Impact of a hospital improvement initiative in Bangladesh on patient experiences and satisfaction with services: two cross-sectional studies

Background The Bangladesh government implemented a pilot Hospital Improvement Initiative (HII) in five hospitals in Sylhet division between 1998 and 2003. This included management and behaviour change training for staff, waste disposal and procurement, and referral arrangements. Two linked cross-sectional surveys in 2000 and 2003 assessed the impact of the HII, assessing both patients' experience and satisfaction and public views and use of the hospitals. Methods In each survey we asked 300 consecutive outpatients and a stratified random sample of 300 inpatients in the five hospitals about waiting and consultation time, use of an agent for admission, and satisfaction with privacy, cleanliness, and staff behaviour. The field teams observed cleanliness and privacy arrangements, and visited a sample of households in communities near the hospitals to ask about their opinions and use of the hospital services. Analysis examined changes over time in patients' experience and views. Multivariate analysis took account of other variables potentially associated with the outcomes. Survey managers discussed the survey findings with gender stratified focus groups in each sample community. Results Compared with 2000, an outpatient in three of the hospitals in 2003 was more likely to be seen within 10 minutes and for at least five minutes by the doctor, but outpatients were less likely to report receiving all the prescribed medicines from the hospital. In 2003, inpatients were more likely to have secured admission without using an agent. Although patients’ satisfaction with several aspects of care improved, most changes were not statistically significant. Households in 2003 were significantly more likely to rate the hospitals as good than in 2000. Use of the hospitals did not change, except that more households used the medical college hospital for inpatient care in 2003. Focus groups confirmed criticisms of services and suggested improvements. Conclusion Improvements in some aspects of patients' experience may have been due to the programme, but the decreased availability of medicines in government facilities across the country over the period also occurred in these hospitals. Monitoring patients’ experience and satisfaction as well as public views and use of hospital services is feasible and useful for assessing service interventions.


Background
The Government of Bangladesh implemented a five year Health and Population Sector Programme (HPSP) from 1998-2003 [1], aiming to improve health services. Despite expansion of health infrastructure and some improvements in health indicators since 1971 [2], in 1998 health services indicators in Bangladesh were poor [3]. Only 45% of the population were covered for essential health care in 1995, against a target of 80% set in 1990 [4]. A 1997 report noted some improvement in access to health services but persisting poor quality of the services [5]. A national survey conducted in early 1999 found only 13% of households used government health services for treatment of recent illness, while 32% used private services; only 37% of households had a good opinion of government health services; 53% of services users rated the service received as good [6]. Outpatient attendance in public hospitals fell from 23.50 million in 1993 to 15.65 million in 1996, with low bed occupancy in secondary care facilities but almost 100% bed occupancy in tertiary hospitals [7,8]. A study in the capital, Dhaka, concluded quality perceptions were driving patients away from public hospitals to private hospitals [9].
The HPSP aimed to improve health services especially among women, children and the poor, through clientcentred provision of an Essential Services Package (ESP), mainly of preventive services, plus selected treatment services. One component was intended to improve the management and quality of hospital services. A pilot Hospital Improvement Initiative (HII) began in 1998 in four district hospitals in Sylhet division and the Sylhet medical college hospital [10]. The HII programme in the five pilot hospitals had several different elements. Formal training sessions aimed to build management skills for hospital board members and managers. Other sessions for all hospital staff with patient contact aimed to help them to improve their interaction with patients, giving them more time and explanations, and respecting their privacy. A new system was introduced for waste disposal, separating clinical and non-clinical waste, and staff were trained in the use of the system. Some structural changes were made in the hospitals to improve privacy (such as construction of separate toilets for women) and equipment such as screens was provided. Systems for procurement were reviewed and updated, and relevant staff were trained in the new systems. Part of the effect of this should have been to improve availability of medicines. The programme initially intended also to improve the referral system, reducing self-referrals and use of these tertiary facilities for primary care. In practice, this element did not happen. The HII programme set up a Health Advisory Committee for each pilot hospital, including community representatives, with the intention of integrating the "citizens' voice" into hospital management.
The HII commissioned a baseline survey in 2000 and a repeat survey in 2003. Designed in consultation with the HII programme team, the aim of the surveys was to evaluate the impact of the HII programme interventions on relevant aspects of patient experiences and satisfaction with services from the hospitals, by collecting data from hospital patients and also from households in communities served by the hospitals. The decision to evaluate the HII programme from the point of view of patients reflected an underlying aim of the programme to make services more client-centred. The surveys were not intended to evaluate the processes of the HII programme; they focused on the experience and views of patients and households at two time points. We shared the findings of the baseline survey in 2000 with local stakeholders, including the management teams of the pilot hospitals and the team implementing the HII programme, and in 2003 we shared the findings from the follow-up survey with the same stakeholders. We prepared technical reports on both surveys [11,12].
The two surveys used the social audit approach. This approach collects evidence (quantitative and qualitative) about services from the people that are meant to benefit from them, and puts this together with evidence from the services themselves, eventually with a view to improving services and helping communities to determine their own health and well being. The philosophy and methods of social audit are described in detail elsewhere [13,14]. The surveys measured the experience and satisfaction of patients visiting the hospitals and documented use of the hospitals and public views about them.
Research on patients' satisfaction dates from the late 1960s [15]. During the 1990s patients' assessment of quality of care became prominent, especially in industrialized countries, and patients' satisfaction with health services became an important area of inquiry [16][17][18]. However, satisfaction surveys have been criticised for being methodologically weak and focussing on the agenda of clinicians and managers rather than patients [19]. Systematic measurement of patients' experiences may be a more sensitive indicator of empowerment [20]. Researchers in the USA developed and tested a standardised instrument to measure patients' concerns and experience. The tool was used in the USA in telephone interviews with hospital inpatients and relatives [21], and since then it has been used in Australia, Canada [22] and various European countries [23,24]. In many developed countries, assessing client satisfaction has been institutionalized as a tool for quality assessment of hospital performance and is used frequently at national and sub-national levels [25,26]. There is an extensive literature using patient or client satisfaction with medical services as a measure of quality [27,28]. However, few studies have made international comparisons of hospital performance [29]. Since 2000, international agencies such as the World Health Organization (WHO) and the Organization for Economic Cooperation and Development (OECD) have stressed patient outcomes as a key indicator of system performance [30,31]. In 2000, WHO included an index of responsiveness to the expectations of consumers in its report on health systems around the world [32]. In 2003, the Health Evidence Network report included consumer surveys among four standard methods of measuring hospital performance [33]. Around the same time, the WHO regional office for Europe initiated a project to provide member states with a practical tool to monitor quality of hospital services [34,35]. The tool was piloted in 55 hospitals, all in developed countries [36]. In developing countries, use of patients' satisfaction as a measure of quality of health services remains relatively rare. Some studies have used patient satisfaction as an indicator of quality of services and some have examined the association between perceived quality of care and health seeking behaviour of patients [9,[37][38][39]. We are not aware of any studies measuring both patient experiences and satisfaction and public views and use of services in order to assess the quality of hospital services.
In this paper we use data from our two surveys to examine formally the changes, over the period of the HII programme, in aspects of patients' experience and satisfaction with hospital services relevant to the elements of service addressed by the programme, and in household views and use of the hospitals, as a way of assessing the potential impact of the programme on these outcomes.

Sample
We selected a household sample and outpatient and inpatient samples for each of the five pilot hospitals: Sylhet Medical College Hospital (MCH), Sylhet District Hospital (DH), Sonamganj DH, Moulvibazaar DH, and Hobiganj DH. Based on the latest census and local knowledge, in 2000 we mapped communities within a five kilometre radius of each hospital. Using random number tables, we selected one urban community within a kilometre of each hospital and one peri-urban community between one and five kilometres of each hospital. In each community, field teams interviewed about 120 contiguous households, radiating from a randomly selected starting point. We revisited the same communities in the 2003 survey.
For each district hospital, the field team interviewed consecutive outpatients as they were leaving the hospital. In the medical college hospital the teams interviewed 100 outpatients from different medical and surgical specialities in proportion to their client load. The inpatient sample was stratified into male and female wards, and into different specialities.

Instruments and data collection
We designed the data collection instruments in consultation with the team implementing the HII programme, aiming to collect information about aspects of patient experiences and satisfaction with the hospitals that would reflect the different elements of the HII programme intervention. Table 1 shows the elements of the programme and the aspects of patient experience we linked to each aspect, recognizing there would be considerable overlap.
The household questionnaire enquired about the demographics and socioeconomic status of the households, and their opinion and use of the DH or MCH.
The questionnaires for hospital outpatients and inpatients asked about age, sex, and socioeconomic status, and included questions about use of an agent (to secure admission), waiting and consultation time, availability of medicines, and payments for aspects of their care. It asked patients about their satisfaction with aspects of their care.
Using a standardised institutional review proforma, the field teams recorded their observations of aspects such as cleanliness and privacy arrangements in the hospitals. We developed the instruments in English, and then translated them into Bangla, with back translation to check preservation of meaning. We piloted all the instruments to check for interpretation and flow.
For each survey we recruited and trained two teams of interviewers from Sylhet town. All supervisors and many of the interviewers from the 2000 baseline survey also participated in the 2003 survey. We contacted the hospitals ahead of each survey to secure their permission to proceed, but as far as possible we did not inform them in advance of the actual day when the team would visit to conduct the surveys.
After preliminary analysis of the surveys at each time point, we developed a focus group guide to share and discuss the main findings from the surveys, and to explore the views of the participants about potential solutions for the problems identified. Field teams returned to the 10 community sites (two for each hospital) and held separate male and female focus group discussions: 20 focus groups in all. A facilitator led each discussion and a reporter took detailed notes, later preparing a report. A small group from the research team reviewed the reports from the focus groups to identify common themes arising around the different topics discussed and to extract illustrative quotes.

Analysis
Data entry operators used Epi-Info to enter data twice with validation to eliminate keystroke errors. Further checks sought logical inconsistencies and out of range responses. Analysis relied on CIETmap open-source software [40] which incorporates an interface to the widely used public domain R statistical software [41].
We calculated the proportions of household respondents and of hospital outpatients and inpatients with different opinions and experience of services, both in the 2000 survey and in the 2003 survey. We tested the significance of changes between the 2000 and 2003 surveys using the Mantel Haenszel procedure [42]. Due to important heterogeneity between hospitals in the changes between surveys, we tested for time changes in each hospital separately. For outcomes that changed significantly between the two surveys in bivariate analysis, we constructed multivariate models to examine whether this change over time could be explained by other variables related to the outcome. We began with saturated models including all the measured variables we believed a priori to be potentially related to the outcome and stepped down to final models that included only variables that remained significantly associated with the outcome. In addition to the time variable, the initial models included age, sex and literacy of the respondent and household head, household size, economic status indicated by the building structure (concrete and blocks, or mud/thatch) and poverty status (monthly household income Taka 5000 (US$ 83 in 2003) or less), and sex of the service provider. Associations are expressed as the adjusted OR (ORa) and 99% confidence interval (CI). We required the 99% CI in order to allow for multiple significance testing.
The research was approved by the CIETinternational ethical review panel.

Results
The sample populations Just under half the inpatients and outpatients were female in both surveys, less than half were literate, and nearly all were from households with a monthly income of Taka 5000 (US$83 in 2003) or less ( Table 2). About half the inpatients and nearly all the outpatients were from the town where the hospital was located. Most of the households surveyed were male-headed, just over half the household heads were literate, three quarters of the household respondents were women, and over two thirds had a monthly income of Taka 5000 (US$ 83) or less. About half the households had a roof constructed of mud or thatch (indicating poverty) ( Table 3). Table 4 summarizes the results from the multivariate analyses of changes in outpatients' experience and views between 2000 and 2003. An outpatient was more likely to be seen within 10 minutes in 2003 than in 2000, in Sonamganj and Moulavibazar DH, and in these same hospitals an outpatient was more likely to report being seen by the doctor for at least five minutes in 2003 than in 2000. Deterioration in medicine availability was significant in three of the DH but not in the MCH. In Hobiganj DH, an outpatient was more likely to receive all prescribed medicines in 2003 than in 2000.

Views and experience of hospital outpatients
Although the satisfaction of outpatients appeared slightly higher in 2003 than in 2000 (Table 4), for most aspects of satisfaction the changes over time were not statistically significant. An outpatient in Sylhet DH was more likely to be satisfied with privacy in 2003; and outpatients in Hobiganj DH were more likely to be satisfied overall with the service in 2003. Table 5 summarizes the results from the multivariate analyses of changes in inpatients' experience and views   DH was there a screen in some consulting rooms providing privacy for examinations. On the wards there were no curtains between the beds or any portable screens to allow privacy when examining patients. In 2003 the teams noted some new brick partitions between groups of beds in three of the DH but no screening for individual beds.

Community views and suggestions
Focus group participants were not surprised by the results from the surveys. When considering ways of improving cleanliness, most felt that hospital management should be responsible for improving cleanliness, for example, through employing and training more cleaning staff. However, they recognized that patients also had a responsibility to help keep hospitals clean, for example by not spitting pan, not dropping waste in and around the premises, using waste bins where provided, and not bringing too many attendants with them. Despite the intention of the HII programme to improve privacy, this was not apparently achieved. Both male and female groups were concerned about continuing lack of privacy in the hospitals. In order to improve privacy, particularly for female patients, they suggested that men should not be allowed to enter female wards or use female toilets.
Presented with the information about the high number of attendants with patients, they recognized that the presence of several attendants for each patient hampered the work of the hospital staff, and posed hygiene and privacy problems, but they pointed out that patients needed the attendants because they did not get adequate personal care from the hospital staff.

Discussion
In some of the hospitals, there was some improvement in patients' experience with administrative aspects of service quality such as shorter waiting times, longer consultations with the doctor, and not requiring the support of external agents to secure admission. There were also some improvements in patients' satisfaction with cleanliness and with staff. The improvements were patchy and did not occur in all the hospitals. No one hospital had improvements in all aspects measured. The MCH did relatively well, especially for inpatient care, while in Hobiganj DH some aspects of patient experience apparently deteriorated. Public opinion about the hospitals in their catchment areas improved over the period of the HII. A limitation of our study is that we did not compare the HII programme hospitals with hospitals not in the programme and therefore we cannot confidently attribute to the HII any changes we detected. Improvements   Over the period, household use of government health services, including hospitals, fell, and the opinion of households and service users about government health services deteriorated [45]. Our findings of increased use ofthe hospitals (although non-significant) and improvements in some aspects of patient experience in the hospitals run counter to the prevailing trends; this strengthens the argument they may be related to the HII programme.
We asked patients about their satisfaction with the care they received. Patient satisfaction ratings can be importantly affected by factors such as culture, expectations, and sense of entitlement [16,46,47]. However, since we revisited the same communities and hospitals in the two surveys, it is unlikely that big shifts in these factors explained the differences we found over time. Allowing patients to judge and evaluate care using their own standards empowers the patient and fosters patient centred care [48]. Ratings may be prone to errors if there is a delay between experience and data collection [49], so most studies rely on contacting patients at the health facility [50]. However, responses to questions on general opinion and service expectations may be biased by the actual service received [51] and consulting only service users excludes the views of people who do not use the services even though they may need them [13].
An important aspect of service delivery deteriorated in most of the hospitals during the HII programme; in 2003 outpatients were less likely than in 2000 to report they received all the prescribed medicines from the hospital. National surveys over the same period reported a significant fall in the proportion of users of government health facilities who received all the required medicines, and non-availability of medicines was associated with lower patient satisfaction [45]. There was also no reduction in the high proportion of inpatients (half or more) with two or more attendants staying with them. The HII apparently did not effectively tackle these problems. Other reports have noted shortcomings of the HII programme. A World Bank report in 2005 concluded there had been little progress towards meaningful autonomy and local accountability of managers in the HII pilot hospitals (which by then included others in addition to the five we studied) and noted the overall lack of benchmarks for evaluating the programmes [52]. Another report noted lack of effectiveness of the Health Advisory Committees set up under the HII programmes to include the community voice in management of the hospitals [53].
Other studies in Bangladesh suggest that even if problems of access could be resolved, quality factors would still influence patients' use of services [54]. Increased dissatisfaction of the public and service users is accompanied by decreasing use of government health facilities and an increase in the unmet need for health care, especially among the more vulnerable [45][46][47][48][49][50][51][52][53][54][55].

Conclusion
The social audit provided useful information on the effects of the HII programme from the perspective of service users and households in the hospital catchment areas. Some improvements occurred in the HII hospitals over the period of the programme, in the face of general deterioration in views and experience of government health services nationally over the same period. The programme failed to prevent the general reduction in availability of medicines in government facilities that occurred over the period. Repeated assessments of patients' ratings of care in hospitals together with documentation of potential users' view and use of hospital services could be a useful part of efforts to improve the quality of hospital services in Bangladesh.