Primary health Centres’ performance assessment measures in developing countries: review of the empirical literature

Background It is universally accepted that primary healthcare is essential for achieving public health and that assessment of its performance is critical for continuous improvement. The World Health Organization’s (WHO’s) framework for performance assessment is a comprehensive global standard, but difficult to apply in developing countries because of financial and data constraints. This study aims to review the empirical literature on measures for Primary Health Centre (PHC) performance assessment in developing countries, and compare them for comprehensiveness with the aspects described by the WHO Framework. Methods Research articles published in English scientific journals between January 1979 and October 2016 were reviewed systematically. The reporting quality of the article and the quality of the measures were assessed with instruments adapted for the purpose of this study. Data was categorized and described. Results Fifteen articles were included in the study out of 4359 articles reviewed. Nine articles used quantitative methods, one article used qualitative methods exclusively and five used mixed methods. Fourteen articles had a good description of the measurement properties. None of the articles presented validity tests of the measures but eleven articles presented measures that were well established. Mostly studies included components of personnel competencies (skilled/ non-skilled) and centre performance (patient satisfaction/cost /efficiency). Conclusions In comparison to the WHO framework, the measures in the articles were limited in scope as they did not represent all service components of PHCs. Hence, PHC performance assessment should include system components along with relevant measures of personnel performance beyond knowledge of protocols. Existing measures for PHC performance assessment in developing countries need to be validated and concise measures for neglected aspects need to be developed. Electronic supplementary material The online version of this article (10.1186/s12913-018-3423-0) contains supplementary material, which is available to authorized users.


Background
During the last three decades, significant achievements have been made in improving the health of the world population [1,2]. This can be attributed in part to the Millennium Development Goals [1] and is further augmented by the launch of the Sustainable Development Goals in September 2015 [2]. In developing countries, however, still more progress needs to be made. In these countries, still far too many women die during childbirth [1][2][3][4][5], too many children die from preventable causes [1,2,4,6] and too many adults die from treatable infectious and non-communicable diseases [6,7]. Reducing mortality and morbidity is the main focus of primary healthcare [1,2]. While progress has been made in communicable diseases, the burden of non-communicable diseases is straining developing countries' healthcare resources [6,7].
Primary healthcare is an essential and critical type of healthcare delivery that addresses the health needs of the population usually delivered at centres called Primary Health Centres (PHC) [8]. In the last decade, the achievements of healthcare have gained significant attention as the 'performance of healthcare' which is not specific to PHC [9][10][11][12][13][14]. In order to monitor their health system, countries carry out performance assessment; the performance is fulfilling one's obligation, in a way that releases one from liabilities [15]. Performance assessment can be defined as a 'coherent evaluation system which assesses the whole occupational functioning including its constituent parts' [16]. A comprehensive assessment of the system in developing countries is vital for determining the gap between demand for services and the ability of the healthcare systems to reciprocate.
In this regard, the World Health Organization's (WHO) health systems performance assessment framework serves as the global benchmark. According to the framework, health systems performance objectives are good health, responsiveness and fair financial contribution [17]. The framework describes six aspects of performance assessment: overall level of health considering the general health of population; distribution of health in population (healthcare services coverage); overall level of responsiveness indicated by the quality of care, satisfaction of care and availability of services; distribution of responsiveness; distribution of resources such as human resources and care facilities; distribution of financial contribution from various agents and their optimal use [17]. Even though it would be ideal, many developing countries might lack adequate resources and data to assess PHC performance according to this framework, and hence, may have reservations about applying the framework [10,18].
The specific measures of PHC performance assessment which are used in developing countries are not well known. In the most recent review of health care performance measurements, no distinction was made between developed and developing countries [19]. The Primary Healthcare Systems are evolving at a faster pace in developing countries than before and lacks clarity, hence the Primary Health Care Performance Initiative calls the system's performance a "Black Box" and identifies an urgent need to build on the existing knowledge [20].
The aim of this study is to review the scientific literature on measures of PHC performance assessment used in developing countries and to compare them with the WHO framework for health care performance assessment for comprehensiveness. Based on the literature, four aspects of healthcare performance measurement [9,11,21] are emphasized in this review: 1. The methods for assessing performance in health care [22] 2. The quality of measurement (validity and reliability) [16] 3. The professional actions, that is, the performance of the provider [23][24][25] 4. The levels covered by the measurement: the level of the patient, that of the community, district/state level and/or country level. Concurrently, measures could capture different perspectives such as that of the health care provider or that of the patient [26].

Methods
This paper employs a review of the literature on PHC performance assessment in developing countries. The flow diagram of the selection and search process is depicted in Fig. 1.

Search strategy
An initial search for articles from the following databases was carried out: Hospital/Health care (Pubmed, Embase), Psychology (JSTOR, PsycINFO) and Business (Emerald Insight). Endnote7 software was utilised to download and select articles. As the aim of the review was to include articles on PHC performance assessment in developing countries that are published in English, the search terms included 'primary health centre' or 'health centre' and 'performance assessment' or 'performance' and 'developing country' {(primary health centre or health centre) and (performance assessment or performance) and developing country}. The period of publication for the search was from the year 1979, a year after Alma Ata up to December 2014. Further, an additional search was done from January 2015 to October 2016 to include recent articles. The search terms were tested and Mesh terms were employed during search when available in the database. The search results from the databases identified were combined to eliminate duplicates. The details of the search strategy are given in Additional file 1: Appendices 1 and 2. A similar search was conducted in Google scholar and the articles fulfilling inclusion criteria were included for full review (Fig. 1). The inclusion criteria employed were: a) Empirical articles with measurement of PHC or professional performance (from professional or health centre perspectives); b) The measurements were conducted in a developing country; and c) Papers were published in English Articles that reported only on methodology of research, discussion on performance/performance assessment and review of literature were excluded. The authenticity of this review was ensured by including only published empirical articles, avoiding grey literature and reviews. On the basis of these inclusion and exclusion criteria, the articles were screened first by title, then by the abstract and full article, to select the relevant articles for full paper review.

Quality assessment
Next, the selected full articles were assessed for quality. The complete evaluation of methodological characteristics as suggested by COSMIN and Noben and colleagues could not be applied, as methodologies of the articles selected were not advanced enough [27,28]. Hence, a simpler instrument was developed, based on the quality of the description and the quality of the measurement property (the reliability and validity) itself. It is theoretically possible that the measurement property is weak although the description is good [28]. With our instrument for assessing the quality of description, the articles were scored as:0, if there was no description of the measurement property;1, if only a few points of the measurement property were described (little information); and 2, if there was a detailed description of the measure in the article (good information).
The quality of the measurement property was assessed based on four criteria: 1. The standardization: method of data collection was noted and the standardization by pretesting of the tool and/or the training of researchers was reported. 2. The triangulation of the data collection method and/or data source with significantly similar results. 3. The measure used was well established/widely published, this was done by verifying references and other cited publications. 4. The reliability and validity of the measures was tested using statistical methods and compared with a global standard (WHO performance assessment framework) [29].
The quality was scored as follows: -0 if none of the above criteria was fulfilled -1 if only standardization of the measure and/or researchers by training was done The results were relegated to a lower number if the higher criteria were not fully met or exclusively mentioned in the article. The qualitative article was scored between 0 and 2 as its measures would be unique, based on the theoretical framework used (scores 3 and 4 were irrelevant). The papers were analysed for quality by the first two authors independently, the differences were discussed and agreed upon.

Descriptive analysis
The selected full articles were analysed and reviewed to understand and describe how the performance of PHCs had been assessed in developing countries. All selected articles were studied to understand how they had defined performance and the perspectives of the performance assessment (provider or patient). The topographical and content analysis of measures was done and these were compared with the measures of WHO performance assessment framework for comprehensiveness [29].

Ethics
No ethical approval was needed for this narrative review.

Literature search results
After downloading the articles (4359 articles) using the search strategy described above, duplicates were removed, inclusion and exclusion criteria were applied systematically, which resulted in the selection of eight articles from the initial search and three articles from the additional search for the review. Further, searches from the reference tracking and in Google Scholar resulted in another four articles (Fig. 1). Thus, in total fifteen articles were included for quality assessment and content analysis (Table 1). Table 1 provides an overview of these fifteen articles.

Quality of measures
Fourteen articles had a good description of the measurement properties. The quality of the measures varied. Measures in eleven articles scored 3, two articles scored 2 and two articles scored 1 ( Table 1). None of the measures was tested for validity with good results.

Description of the PHC performance measures
Empirical research on PHC performance measures was published from Brazil, India, Papua New Guinea, Egypt, Bangladesh, Armenia, Malawi, Tanzania, Rwanda, Serbia and other developing countries. A descriptive analysis revealed that PHC performance was distinguished as individual staff/personnel performance and that of the centre/health system performance with a focus on consumer's and/or provider's perspectives. Figure 2 depicts the perspective and level of assessment.

Performance of the professionals
Five studies evaluated the personnel performance assessment (nurses, midwives, management team, community health workers and medical professionals). The assessment considered competency/clinical services and non-competency/non-clinical service-based components. The personnel performance was assessed using the Quick Investigation of Quality tool [34] (Fort and Voltero 2004). Adherence to the protocol was the most common method for competency based personnel performance assessment [30,33,35,36,38]. For the non-competency based personnel performance assessment, the measures used were punctuality, response time, absenteeism, time of reporting to work, time spent with patients, waiting time and cleanliness [33,36]. Reproductive healthcare delivery was evaluated in all the papers [30,33,35,36,38].
It was found that autonomy of the team, availability of manpower, clarity in the job description, roles and responsibilities, working conditions, workload and environment, the level of motivation, their education and training along with good supervisory practices contributed to the performance of healthcare personnel [33,38].

Perspective and structural levels of performance
The target groups in the studies were individual patients, local community, district or country level. Health care personnel such as nurses, doctors and health managers along with other key stakeholders like local governing bodies were included. From the respondent's perspective, performance was defined as satisfaction, accessibility of the centre, case detection and success of the treatment provided in the country, care experiences, establishment of patient's bonds for treatment success and organizational care. Centre performance assessment was assessed as provider and patient bonding leading to therapy compliance and treatment success. This was also assessed in terms of costs and effectiveness of the services provided, the satisfaction of the providers and other stakeholders on the availability of resources and functioning of the centre. The scope of the centre performance assessment was at the structural levels of centre, community, district or country.
Comparison with the WHO aspects of health systems performance assessment The measures described in the articles were compared with the six aspects of the WHO health systems performance assessment framework (Table 2) [29]. In the articles included, measures for three aspects (overall level of responsiveness, distribution of responsiveness and distribution of resources) were considered in relation to specific diseases/services, though the WHO framework uses system-specific measures. Measures for disease/service specific WHO aspects (overall level of health, distribution of health in population and distribution of financial contribution) were less well represented in the fifteen articles.

Discussion
The aim of this review was to describe the measures of PHC performance assessment used in developing countries as published in the empirical literature and to compare them with the WHO framework for health systems performance assessment. The fifteen articles that were considered provided scarce information on measurement quality and covered limited aspects of PHC performance when compared to the WHO health systems performance assessment framework. Measures were found both at the level of the performance of professionals and of the centre, and further measures that addressed the satisfaction of the performance by stakeholders were found. These measures will be discussed below.

Performance of professionals
Personnel performance was based on observation methods, assessed competency and non-competency based tasks. This correlated highly with patient satisfaction, an indicator of centre performance, as well as that of availability of resources, support and culture of the organization [45]. The investigators used the 'Quick Investigation of Quality' tool that had been validated earlier [45,46]. Such validated tools for observation of professional performance provide a quick  • Incidence, prevalence and mortality by disease [42] • Community health [31] • Number of deliveries and maternal mortality figs.

Performance of the Centre
The performance of the centre was assessed using the Primary Care Assessment Tool [40,41,43]. This tool was validated in a developing country for family healthcare [41,47]. It is interesting to note that this instrument could be used to assess the performance from the consumer's as well as from the provider's perspective. This tool provides an extensive list of surveys on different attributes of PHC. Based on the requirement of the evaluation, one could consider specific components. In other studies, measures such as costs, patients served and effectiveness along with the client satisfaction were identified. Cost effectiveness is an appropriate measure that can be used across all PHCs [48] and thus, helps in making right choices by identifying the most effective service or intervention or centre [48].

Satisfaction with performance by various stakeholders
Satisfaction of various stakeholders including patients and providers is an accepted standard in performance assessment as they are interrelated. Community participation in the functioning of the PHC highly influences the performance of PHC and its personnel, so it could be included as a component in the PHC performance assessment [34].

WHO aspects of performance assessment
The current practices of PHC performance assessment in developing countries were analysed with the WHO framework for performance assessment [17,29]. The WHO framework is completely focused on output and outcome, with the structure and process considered intrinsic to the system. However, for ease and clarity, many authors have used the Donabedian model for assessing performance in primary health care [19,23,26,49]. The current review highlights the limited representation of the performance measures in relation to aspects of the WHO framework (Table 2). In the articles, even the tools used to assess personnel performance (Quick Investigation of Quality tool) and centre performance (Primary Care Assessment Tool) did not represent measures from three aspects: overall level of health, distribution of health in the population and the distribution of the financial contribution [40,41,47]. Due to lack of resources and data [10,18], covering the complete list of WHO aspects to the full extent will be difficult for developing countries. However, a concise list of measures with an appropriate representation of six WHO aspects and requiring minimum resources and data, needs to be developed for PHC performance assessment in developing countries. If the assessment uses standardised measures useful comparisons across regions/countries would be possible. Since the search results indicated that a very small number of articles were published, further research needs to be conducted in the developing countries on PHC performance assessment, enabling cross learning and knowledge base enhancement [21].

Discussion on methodological quality of articles
Although the number of articles in this review was limited, the studies covered diverse countries and continents. Assessing the quality of the papers was a challenge as articles used qualitative, quantitative and mixed methodology. The articles demonstrated reliability and validity methods such as correlating provider and user experiences, data/method triangulation, standardised tools/training of researchers. Hence, there is a need for research in developing countries to establish quality of measurement and standardisation of PHC performance assessment using rigorous statistical methods.
Implications for future research PHC performance assessment in developing countries is an emerging field but it is fragmented at present. Though human resource for health is a component of the WHO framework, the various sub-components to be included are not clear. Evaluation of the PHCs is an on-going exercise, but yet there is no established standard for assessing performance. Studies have shown that the performance of PHC depends on several things, one critical factor being personnel performance [21,29,[50][51][52].

Implications for practice
What is measured, can be controlled. However, not all aspects of PHC performance were yet covered. Measures such as skill sets for execution of healthcare focusing on mother and child health, are already well available. They were emphasised in five studies of our review and this is in line with Millennium Development Goals and Sustainable Development Goals [1,2]. However, other aspects like coverage/distribution of health, overall health of the population, distribution of financial contribution, including program specific achievements, various service components and process of primary care delivery are still not covered. The performance measures should be appropriate and adequate enough to enable accurate assessment on an ongoing basis to aid in monitoring and efficient management of the personnel/system. There is urgency to develop new and additional measures for PHC performance in developing countries. PHC performance is a matter of immense importance for policy agenda and political priorities [53].

Conclusions
Although developing countries may have difficulties in applying the entire WHO framework, the current measures for assessment of PHC performance published in scientific journals are limited in scope and lack validation. The standard health indicators for the overall level of health and for the distribution of health in the population were represented least. A comprehensive assessment of primary healthcare can be achieved by integrating personnel performance with that of centre performance. Representation/inputs from both sides of the service delivery, the management and the consumer/ public, that is, including the provider's perspective and consumer's perspective are vital. From this review, it can be concluded that existing measures for PHC performance assessment in developing countries need to be validated and concise measures for neglected aspects need to be developed.