Quality improvement in medicines regulation: a retrospective analysis of the Pharmacy Board of Sierra Leone before and during quality management system implementation

Abstract

Background

National Medicines Regulatory Authorities like the Pharmacy Board of Sierra Leone are responsible for protecting and promoting public health, implementing regulatory standards, and maintaining a supply chain with an assured supply of medical products that are safe, effective, and of good quality. This retrospective study assesses the identification of substandard and falsified medicines, the changes in the functions and key indicators of assessment, and the quality improvement changes of the Pharmacy Board of Sierra Leone.

Methods

Data was obtained from 2013 to 2021 records using a data collection tool to collate and review all relevant information to address the different objectives. All data were sourced from the Department of Quality Assurance and the Department of Enforcement and Narcotics at the Pharmacy Board of Sierra Leone. The review also included, identified substandard and falsified medicines, the World Health Organisation Global benchmarking self-assessment tool, and internal and external audit records of the quality management system of all twelve departments of the Pharmacy Board of Sierra Leone.

Results

The study showed marked changes in identifying substandard and falsified medicines by the Pharmacy Board of Sierra Leone during ISO 9001:2015 implementation (2017- 2020) compared to Pre-ISO 9001:2015 implementation (2013- 2016). Critical functions of the Pharmacy Board of Sierra Leone from the assessment of the WHO GBT ML in 2016 and 2021 showed that several indicators had been addressed during ISO 9001:2015 certification with improvement in the level of maturity for the quality management systems and Pharmacovigilance functions. There was also an improvement in identifying non-conformances and a commitment to continuous improvement of processes during ISO 9001:2015 implementation.

Conclusions

This study revealed that regular checks through standard assessment, internal audits, and standard management review processes that generate follow-up actions, timelines, and a commitment to identifying correction, and corrective actions for non-conformances are essential quality improvement tools for the efficient functioning of an institution (Pharmacy Board of Sierra Leone). Our study revealed that commitment to continuous implementation of proper quality management system could significantly improve institutional efficiency, thereby improving service delivery and customer satisfaction.

Background

Many sub-Saharan African countries continue to face challenges in developing and implementing effective medicines regulatory systems. Challenges such as poor inspection practices, ineffective licensing and product registration systems, inadequate access to quality control laboratories, and weak pharmacovigilance and clinical trials have contributed to the proliferation of substandard and falsified medicines in these markets [1, 2].

Substandard and falsified medicines pose a serious threat to public health in Africa. They cause increased morbidity, higher treatment costs, and poor therapeutic outcomes. They also undermine the national health budget and erode the trust in medicines and the healthcare system. This problem is often underreported and requires urgent attention. The World Health Organisation (WHO) estimates that about 10% of medicines circulating worldwide and 25% in less developed countries are substandard and falsified medicines. Africa and some parts of Asia are the most affected [3, 4]. Africa’s main sources of medicines import, the Chinese and Indian pharmaceutical industries have been classed as the major producers of substandard and falsified medicines [5].

Effective regulation of medicines requires a well-structured system that involves multisectoral collaboration and actions ranging from legal basis to quality evaluation and monitoring [6]. WHO uses the ‘Global Benchmarking Tool Maturity Level (WHO GBT ML)’ to objectively evaluate National Medicines Regulatory Authorities’ (NMRA) performance and to establish their levels of maturity and functionality in regulation [7]. There are four maturity levels adopted from the International Standard ISO 9004, and they are reflections of how stable, integrated, and the extent of functioning of a regulatory system. Maturity level-1 represents a low maturity and functionality, while Maturity Level-4 is a higher maturity level attained by NMRA with advanced and efficient functionality. In 2010, reports from WHO revealed that only 7% of 46 sub-Saharan African countries have moderately developed medicine regulatory capacity, 63% have minimal capacities, and 30% have National medicines regulatory authorities in place [8]. There is no maturity level-4 NMRA operating in Africa. Nigeria, Ghana, South Africa, and Tanzania are currently operating at maturity level 3 while the NMRA of Sierra Leone was classified using the WHO-self benchmarking tool in 2016 before the establishment of a quality management system, was classified at an overall maturity level-1 [9, 10], and for all functions.

One of the key functions of NMRAs is to evaluate the safety, effectiveness, and quality control data of new drugs before granting marketing approval [11]. However, this process can be lengthy and inefficient, resulting in delays in patients’ access to affordable and essential medicines [12]. Different regulatory authorities have varying requirements, structures, and processes for drug development, which makes it hard for pharmaceutical companies to develop drugs for simultaneous submission to all regulatory authorities [13, 14]. In this Global competitive environment, reducing the time taken by a product to reach the market is a critical parameter; hence, the company’s success relies on that. The South African NMRA typically receives about 4,700 applications annually but can only process around 2,550 per year [15]. Before 2005 there was an equilibrium between the number of applications received and the registration certificates issued for medicines per year. However, from 2005 the number of applications submitted increased significantly whereas the number of certificates issued remained approximately the same [12].

There is a need to improve the regulatory capacity and performance of NMRAs in sub-Saharan Africa by implementing quality management systems (QMS) that can ensure consistent and transparent standards and procedures. QMS are defined as “coordinated activities to direct and control an organisation with regard to quality” (ISO 9000:2015) [16, 17]. QMS can help NMRAs achieve their objectives by providing a framework for planning, implementing, monitoring, evaluating, and improving their processes [17]. Effective and efficient regulation of medical products could provide an opportunity for investment in manufacturing, trade, and sale of pharmaceutical products, as well as an increase in research and development of new medical products and technologies. These yield social and economic benefits to the patients and communities. Building the regulatory capacity of medical products is also crucial for achieving Universal Health Coverage (UHC), the Africa Union (AU) Agenda 2063, and sustainable development goals (SDGs) on access to quality, safe, and efficacious health products.

The Pharmacy Board of Sierra Leone is the National Medicines Regulatory Authority and the first government institution in Sierra Leone to have certification for its quality management system (ISO 9001:2015) in 2019. It has vital departments, including Quality Assurance (Quality management system), Pharmacy Practice and Licensing, Quality Control Laboratory, Information and communication, Drug Evaluation and registration, Inspection (factory and import inspectorate and the distribution chain inspectorate), Pharmacovigilance and Clinical trials, Enforcement and Narcotics Control and Complementary and Alternative medicines in Sierra Leone. This retrospective study assesses the number of identified substandard and falsified medicines, the changes in the functions and key indicators of the Pharmacy Board of Sierra Leone using the WHO GBT maturity level, and identified the quality improvement changes of the organization as per ISO 9001:2015 at different stages of the quality management system. While the implementation of quality management system is a commendable step, it is essential to evaluate its impact on regulatory functions and overall performance in a resource-limited setting like Sierra Leone. The findings of this study provides an understanding of the role of quality management systems in enhancing the efficiency of national medicine regulatory authorities, inform in-country policy decisions and guide regulatory bodies that are in pursuit of continuous quality improvement.

Methods

Study setting

This study was conducted at the Pharmacy Board of Sierra Leone (PBSL), located in Freetown, West End Area Urban, the National Medicines Regulatory Authorities (NMRA) in Sierra Leone, and has been certified for good quality management system (ISO 9001:2015) since 2019. The PBSL is responsible for regulating pharmaceutical products, medical devices, cosmetics, chemicals, and other related substances in Sierra Leone.

Study duration

This study lasted for a period of 4 months, from July to October 2021. The data reviewed was from the year 2013 to 2021.

Study design

A Retrospective study was done to assess the changes in the quality improvement of the processes and services to customers of the Pharmacy Board of Sierra Leone (PBSL) and the effect of internal and external audits on improving service delivery. Data was collected from reports and records of identified substandard and falsified medicines, internal and external audit reports, and the WHO global benchmarking tool from 2013 to 2016 (Pre-QMS implementation), and from 2017 to 2020 (during QMS implementation). All quality documents related to the objectives of this study were reviewed.

• Pre-QMS implementation (pre-QMS): the period before QMS implementation as per ISO9001:2015. This period was from 2013 to 2016.

QMS implementation (QMS): the period during which commitment was established for the implementation of QMS, during the international certification of PBSL in 2019 and maintenance of QMS in 2020. This period was from 2017 to 2020. This period can also be divided into two to include.

• a) 2017/2018: QMS implementation as per ISO 9001:2015, without certification: QMS_nc.

• b) 2019/2020: QMS implementation per ISO 9001:2015, with international certification achieved in 2019 and maintained in 2020: QMS_c.

Data collection

The data collection tool was prepared and used to collate relevant information from records and reports at department of Quality Assurance and the department of Enforcement and Narcotics at the Pharmacy Board of Sierra Leone. The internal quality audit records and quality audit reports from external auditors AFNOR, WAHO, USP, and the WHO Global benchmarking self-assessment report were all reviewed.

The WHO Global Benchmarking Tool (WHO GBT) for maturity level is a QMS tool derived from the ISO Maturity model for assessment and improvement that objectively assesses the level of maturity and functionality of National Medicines Regulatory Authorities. It is an efficient tool that identifies strengths, and areas of improvement, facilitates the formulation of institutional development plans (IDPs), and provides a basis to build upon strengths and resolve identified gaps while prioritizing IDP interventions and monitoring progress and achievements.

The WHO global benchmarking tool is a standard self-assessment checklist that calculates the percentage implementation of regulatory systems and score key specified regulatory activities from level 1(low-level regulatory performance) to level 4(high-level regulatory performance). The WHO global benchmarking tool is available on the WHO website and the WHO started using this tool to assess regulatory systems in 1997. The WHO GBT provides a thorough assessment of the National regulatory system, Registration and marketing, Vigilance, Market surveillance and Control, Licensing processes, Regulatory inspection, laboratory access and testing, Clinical trials oversight, and lot testing.

Data analysis

The data was entered into an excel sheet and thoroughly checked for correctness against the raw data by three independent researchers. The pre-QMS period was determined to be from 2013 to 2016, while the QMS period was from 2017 to 2020. The substandard and falsified medicines and functions and indicators of the National regulatory system were determined and collated in a tabular form based on Pre-QMS and QMS implementation. The percentage of key indicators and percentage change of the key indicators were also determined and collated in a tabular form based on Pre-QMS and QMS implementation. Quality improvement parameters were determined at the Pre-QMS and QMS implementation period.

Results

Substandard and falsified medicines identified Pre-QMS and at QMS implementation

The results presented in Table 1 show the number of substandard and falsified medicines identified by the Pharmacy Board of Sierra Leone before QMS implementation. The results indicate an increase in the number of substandard and falsified medicines detected during QMS implementation compared to pre-QMS implementation.

Table 1 Number of identified substandard and falsified medicines from before QMS implementation and during QMS implementation

Status of the functions of the Pharmacy Board of Sierra Leone using the WHO GBT ML assessment conducted at Pre-QMS and QMS implementation

The results indicated a marked improvement in the Quality Management System during QMS implementation as shown by the % change in most sub-indicators and little or no change in other sub-indicators. A remarkable improvement was observed in the quality management system with a 33% improvement in processes and procedures. This is shown in Table 2.

Table 2 Assessment of pharmacy board functions using self-WHO global benchmarking indicators

Review of key indicators for the national regulatory system using the WHO GBT ML

Table 3 shows a reduction in financial resources to perform activities (100% in 2016 to 95% in 2020), while human resources to perform laboratory testing also dropped (100% in 2016 to 94% in 2020). Well-maintained and equipped infrastructures for laboratory testing also decreased (100% in 2016 to 75% in 2020) while measures for occupational health and safety also decreased (66.7% in 2016 to 50% in 2020).

Table 3 Key indicators for national regulatory system from the WHO GBT ML of 2016 and 2021

Quality management systems including how risk management principles are applied and realized increased (28.6% in 2016 to 98.0% in 2020) and mechanisms for promotion of transparency, accountability, and communication increased (60.8% in 2016 to 92.0 in 2020). Regulatory systems that is supported with leadership and crisis management plans also increase (40.0% in 2016 to 100% in 2020).

Assessment of the status of quality improvement parameters from the records of the WHO GBT ML and the internal and external audit reports

Table 4 shows the status of the quality management system from 2016 to 2020 at pre-QMS implementation and during QMS implementation.

Table 4 Findings from quality improvement parameters based on the WHO GBT ML and internal and external audit

Discussion

Quality improvement (QI) consists of systematic and continuous actions that lead to measurable improvement in healthcare services and the health status of targeted patient groups or services. The Institute of Medicine (IOM), a recognized leader and advisor in improving the Nation’s health care, defines quality in health care as a direct correlation between the level of improved health services and the desired health outcomes of individuals and populations [18].

Our study provides the changes in the identified counterfeit medicines by the Pharmacy Board of Sierra Leone, the status of the functions and key indicators and quality improvement changes at Pre-QMS and at QMS implementation using reviewed data at the Quality Assurance and the Enforcement and Narcotics Department of the Pharmacy Board of Sierra Leone.

Substandard and falsified medicines identified Pre-QMS and at QMS implementation

Our result showed an increase in the number of identified substandard and falsified medicines (unregistered drugs with the Pharmacy Board of Sierra Leone) during QMS implementation for ISO9001:2015 compared to the pre-QMS implementation. This increase in the number of identified substandard and falsified medicines could be due to the implementation of the post-market interventions for identifying substandard and falsified medicines and the improvement in the quality management system at the Pharmacy Board of Sierra Leone. Antibiotics were also observed as commonly identified substandard and falsified medicines. This could be the result of the high demand for antibiotics in Sierra Leone and the quest by counterfeiters to make more money. Antibiotics are the most substandard and falsified medicines, accounting for about 28% of drug counterfeiting in 1999 and 2002 [19]. substandard and falsified antibiotics will not cure an infection and may contribute to the global crisis of antibiotic resistance if left unattended [20, 21]. The increase in substandard and falsified medicines may also have been because of the prioritization of funding from other Ministries, departments, and agencies towards supporting the COVID-19 pandemic in 2020.

National medicines regulatory authorities and governments worldwide have prioritized ensuring effective pharmaceutical regulation by enhancing regulatory mechanisms and improving regulatory performance. This is achieved by enforcing strict regulatory requirements and ensuring a steady supply of safe, effective, high-quality medical products [6]. Despite the vital role that NMRAs play in national healthcare systems, medical product regulation is frequently under-recognized and underfunded in most countries. It is known that at least 30% of NMRAs cannot conduct critical regulatory functions and are without enough financial support for product regulation [22, 23].

Status of the functions of the pharmacy Board of Sierra Leone using the WHO GBT ML assessment conducted at pre-QMS and QMS

This study showed marked improvement in the quality management system of the Pharmacy Board of Sierra Leone during QMS implementation. Such improvement could have resulted from the commencement of QMS documentation in 2017, attaining certification for QMS in 2019 and maintaining certification 2020. All over the world, different organizations are known to implement requirements of the ISO9001:2015 standard to demonstrate their ability to consistently provide products and services that meet customer and good regulatory requirements. The quality management system standard (ISO9001:2015) is the most popular standard of the ISO9000 series and to which organizations can be certified [17]. Marketing Surveillance and Laboratory Control attained maturity level 2 after certification in 2019 and this could be attributed to the strict implementation of internal and external audit recommendations in 2019. The PBSL was initially assessed by the West African Health Organization (WAHO), Quality Catalyst consultancy, and the Quality Assurance Department (internal audit) of the Pharmacy Board of Sierra Leone before the certification assessment audit in 2019. Most of the recommendations from the audits before 2019 were geared towards strengthening the quality management system (regulatory system) in the office and taking action to ensure strict adherence to internal procedures and the ISO9001:2015 standard. These audits could have helped prepare the PBSL to address all the identified non-conformities and opportunities for improvement and allow for proper preparedness in readiness for the certification audit by AFNOR as per ISO9001:2015 in the year 2019. All other functions showed minimal change in the percentage of sub-indicators addressed, with no change in the maturity level. This is possibly because of the need to address several gaps in the law (Pharmacy and Drugs Act 2001) and or prepare regulations of the Pharmacy Board of Sierra Leone that could improve regulatory activities and subsequent attainment of higher maturity levels (level 3 or 4) as per the WHO global benchmarking self-assessment tool.

Review of key indicators for the national regulatory system using the WHO GBT ML

Our study showed a reduction in financial resources to perform activities (100% in 2016 to 95% in 2020), reduced human resources to perform laboratory testing (100% in 2016 to 94% in 2020), reduced infrastructures for laboratory testing (100% in 2016 to 75% in 2020), and in the measures for occupational health and safety (66.7% in 2016 to 50% in 2020). These reductions could be due to the initial implementation of the single treasury account and the reassignment of financial resources towards the recent COVID-19 pandemic. Without a doubt, there is need for urgent prioritization of resources to the laboratory of the Pharmacy Board of Sierra Leone and the expression of interest for implementation of the quality management system for testing laboratories (ISO17025).   

An improvement was observed in the application of risk management principles at the office (28.6% in 2016 to 98.0% in 2020), improvement in existing mechanisms to promote transparency, accountability, and communication (60.8% in 2016 to 92.0 in 2020), an improved regulatory system that is supported with leadership and crisis management plans (40.0% in 2016 to 100% in 2020). These improvements could all be attributed to the fact that staff was assigned the responsibility of taking action for the establishment of processes and procedures for implementing a quality management system in 2017. This was later followed up with a fully established Quality Assurance department and a head of quality assurance for the PBSL in 2019.

Assessment of the status of quality improvement parameters from the records of the WHO GBT ML and the internal and external audit reports

Most key quality processes and documents were unavailable or not approved in 2016, although active efforts to establish good coordination for implementing international norms and standards. In 2017, draft processes and documents were available but needed to be approved. Most of the required documents and processes for sound quality management systems were available and approved in 2019. These improvements occurred because of the implementation of key recommendations from the internal audit by the Quality Assurance department and the external audit from partners (WAHO, Quality Catalyst, Quality Assurance internal audit) that identified gaps in documentation and process. The gaps addressed before the expression of interest for certification gave confidence to staff and prepared the Pharmacy Board of Sierra Leone for certification in 2019. This showed that implementation of a good quality management system requires the use of sound principles of quality management system by use of management review, internal and external audit procedures of measurement and monitoring while adhering to appropriately prepared standard operating procedures, guidelines, ISO9001 standards, and international best practices using the Deming’s cycle for development of organizational objectives and strategic plan development for continuous quality improvement. 

In addition, to address the issues faced by National Medicines Regulatory Authorities (NMRAs) in resource-constrained environments, efforts have focused on identifying and performing essential regulatory duties that must be carried out directly by NMRAs to meet country or regional needs through regulatory convergence that allow for interaction with and appreciation of the efforts of other regulators to reduce regulatory burdens by collaboration and dependence [9, 15]. Our research is a confirmation that expression of interest and implementation of quality management system and or including certification for WHO prequalification could serve as a facilitator for improved service delivery, improved medicines safety, regulatory efficiency and in assuring customer satisfaction particularly in low-income countries.

Thus, establishing the principles of a sound quality management system in an organization or national medicine regulatory authority can improve regulatory services and customer satisfaction for the provision of safe, efficacious, and good-quality medicines [2, 15, 24].

The strength of our study is the use of a standard validated tool for assessment of quality management system of regulatory authorities globally. The tool also helped in addressing recall and observer bias as a result of memory and self-reported data. The limitation of our study is that it provides information on the QMS of one medicine regulatory authority and in a low-income country.

Conclusion

Our study revealed that regular checks through self-assessment and internal audits are essential quality improvement tools for the efficient functioning of the Pharmacy Board of Sierra Leone. Identifying gaps in the management system using the principles of quality assurance, risks, root cause analysis, correction, and corrective action and implementing appropriate mitigation measures could resolve deficiencies in the regulatory system, improve performance and thereby contribute to better stakeholder satisfaction. This study also showed that regular planning, management review, and resolution of issues arising from monitoring and measurement and internal and external audits by taking action for improvement (Deming’s cycle) are important tools for a sound quality management system. It is therefore without a doubt that prioritization of a Quality Management system could serve as an effective means for improving regulatory outcomes and client satisfaction in low, middle, and high-income countries.

Annexes: Data collection guide

Questions	Information
Number of substandard and falsified medicines identified per year	-PBSL annual report -Lab annual report for failure in quality analysis
Status of quality document 2016–2021 as baseline -SOP guideline quality manual	-Quality Assurance
Status of forms
Status of the pharmacy board department Baseline 2016–2021	-Quality Assurance
Internal Audit report for the past five (5) years External agencies	-USP audit report -WAHO internal audit -Royal Consultancy -Quality catalyst -AFNOR surveillance 1 -AFNOR surveillance 2 -AFNOR pre-inspection audit
Internal agencies	-PBSL audit 2016–2021
Status of monitoring and evaluation	-Quality assurance
Management review