1. Introduction
The increasing prevalence of antimicrobial resistance (AMR) is a global public health threat, recognized by the World Health Organization (WHO) member states and the World Health Assembly as a priority area for global action [
1].The AMR Global Action Plan (GAP), introduced by the WHO in 2015, has provided guidance to countries in tackling antibiotic resistance, with microbiological surveillance as a key action [
2].
AMR surveillance is important for revealing patterns, trends, and outbreaks of resistant microorganisms at the national and international levels [
3]. Data from this surveillance allow policymakers and health care providers to introduce evidence-based standards and regulations and to make appropriate decisions about antibiotic prescribing [
4].
Countries are expected to report annually to the WHO regarding their progress on AMR data using the Global Antimicrobial Resistance Surveillance System (GLASS). The GLASS-AMR platform collects aggregated data on the frequency of antimicrobial resistance among high-priority pathogens that cause acute infections in humans, namely,
Escherichia coli, Klebsiella pneumoniae, Salmonella spp.,
Shigella spp.,
Acinetobacter spp.,
Streptococcus pneumoniae, Staphylococcus aureus, and
Neisseria gonorrhoeae. These pathogens are the most commonly identified in hospital-acquired and community-acquired infections, and treatment is becoming increasingly difficult because of high rates of AMR. Moreover, some of the selected bacteria included in the GLASS are present in non-human animals and the food supply. The data for the GLASS are collected at the national level and enable countries to share comparable and validated data worldwide to drive national, regional, and global actions in reducing the AMR burden [
5].
Other than AMR data, the GLASS also is a repository of information about the status of national AMR surveillance systems, including the presence of a National Reference Laboratory (NRL), National Coordinating Centre (NCC), and National Action Plan (NAP) from enrolled countries. By April 2020, 91 countries and territories had enrolled in the GLASS, and 66 countries submit AMR data reports [
5].
Laboratory-based surveillance systems require resources, facilities, training, and a central public health reference laboratory for quality assurance [
6]. The Ministry of Health and Population in Nepal, with the participation of various multi-sectoral stakeholders, has drafted an NAP for AMR in line with the GAP and, in 2017, designated the National Public Health Laboratory (NPHL) as the NRL and NCC for Nepal.
The NPHL initiated a laboratory-based AMR surveillance about two decades ago, with nine sentinel sites [
7], which has now expanded to 21 sites country-wide. Ten of these sites are situated in Province 3, one of the most populous provinces in Nepal [
8]. The NPHL receives AMR data from the surveillance sites and is responsible for collating and combining the data before submission to the GLASS. The completeness, consistency, and frequency of data reporting vary among surveillance sites, with some sending the information monthly, others quarterly, and others only sporadically. The NPHL has adhered to the GLASS initiative and began to submit data to the platform in 2017 [
9].
The aim of this study was to assess and verify the AMR surveillance laboratory data reported to the GLASS from five AMR “reporting” surveillance sites and to use a basic infrastructure assessment to identify barriers to submission for four “non-reporting” sites, all in Nepal’s Province 3. To our knowledge, this work represents the first such assessment of the quality of AMR data submitted through a surveillance system in Nepal.
2. Materials and Methods
2.1. Study Design
This is a cross-sectional descriptive study using secondary AMR data collection and a structured questionnaire.
2.2. Settings
2.2.1. General Setting
Nepal is a landlocked country in Asia with a population of around 30 million. The country consists of seven self-governing provinces with 77 administrative districts and 753 local self-governing bodies. Province 3 (presently Bagmati Pradesh) represents 22% of the total country population. Moreover, it has the highest number of microbiological surveillance sites (10 out of the total 21 surveillance sites in the country), which are easily accessible and are larger sites with a relatively high volume of AMR data. For these reasons, we selected them for this study.
2.2.2. Specific Setting
The NPHL is the national reference laboratory responsible for monitoring, supervising, capacity-strengthening, and providing an external quality assessment (EQA) of all microbiological surveillance sites in the country. The NPHL took on the responsibility of reporting to the GLASS in 2017, when only one site submitted complete data, which was not published in the GLASS. In 2018 and 2019, the reporting sites increased from 14 to 15 out of 21 that provided complete data to the GLASS platform [
5]. The data are supposed to be received monthly from all the AMR surveillance sites and collated and compiled at the NPHL before submission to the GLASS.
The NPHL supports the surveillance sites through annual refresher trainings on “Identification of AMR surveillance pathogens and antimicrobial susceptibility testing”, data entry, deduplication and compilation, and on the use of the WHONET software, a freely downloadable database software developed for the management and analysis of antimicrobial susceptibility test results [
10]. The surveillance site laboratories are trained on standard procedures of antimicrobial susceptibility testing following the annually updated Clinical and Laboratory Standards Institute (CLSI) criteria. The NPHL also regularly supervises the laboratories through periodic monitoring visits, although not all sites are necessarily visited each year. The NPHL further ensures the quality of laboratories by sending regular EQA scheme samples, followed by feedback on the laboratory performance.
Only nine of the ten sites located in Province 3 were considered for this study because one of the sites is an animal health site and thus not eligible for GLASS reporting. The selected surveillance sites are located at tertiary care or specialty hospitals with in-patient as well as out-patient departments in Province 3, except the NPHL which is the reference stand-alone laboratory. Five of the sites sending regular data to the NPHL were identified as “reporting” sites, whereas four that had failed to report data for three consecutive months were identified as “non-reporting” sites.
2.3. Study Population and Period
We reviewed detailed laboratory records of positive bacterial culture of AMR pathogens sent to the NPHL from January to June 2019 from the five “reporting” surveillance sites. The five “reporting” sites had variable numbers of specimens tested varying from 6000 to 36,000 with most being blood and urine specimens. The positive bacterial culture numbers also varied from site to site, so a random subset calculated from previous year’s records was selected for verification. The detailed laboratory records (range 171–428 per site, totaling to 1584 records) from each site, calculated on the basis of 2018 submitted GLASS data (total 89,553 records) through online OpenEpi software, were selected for verification. In addition, one microbiology laboratory staff involved in data compilation from each of the “reporting” and “non-reporting” surveillance sites was asked to complete a paper-based structured questionnaire between 15 January and 15 February 2020.
2.4. Data Variables, Sources of Data, and Data Collection
2.4.1. Variables
The following variables were extracted from the Excel spreadsheet maintained at the NPHL to check the consistency, completeness, and timeliness of data in accordance with the GLASS reporting criteria: unique ID, type of specimen (blood, stool, urine, or genital swab), bacterial isolates (Escherichia coli, Klebsiella pneumoniae, Salmonella spp., Shigella spp., Acinetobacter spp., Streptococcus pneumoniae, Staphylococcus aureus, or Neisseria gonorrhoeae), antibacterial agents, age, sex, date of specimen collection, origin of the specimen, and detailed antibiotic susceptibility report. The overall consistency was assessed for this study by checking the variables used to generate the report (as directed by the GLASS manual i.e., four priority specimens and eight priority pathogens) while completeness signified no missing variables required according to the GLASS criteria: age, sex, pathogen, origin, specimen, antibiotic susceptibility results, and date. The dates that the monthly results were received at the NPHL were used to report the timeliness.
We used a structured questionnaire to collect variables, including human resource availability, AMR/WHONET software training, data analysis training, number of rooms available in the laboratory, dedicated space for data entry, computer and Internet service availability, agreement with the NPHL, availability of terms of reference, and institutional restrictions. As noted, one microbiology laboratory staff from each of the “reporting” and “non-reporting” sites completed the questionnaire.
2.4.2. Sources of Data
The sources of data assessed were the monthly AMR data reports received at the NPHL (formats: Excel document, email, paper-based, or jpg file) and on-site AMR surveillance data from the “reporting” sites (registers/electronic database/software/WHONET file) only as the “non-reporting” sites had not sent the results for ninety consecutive days and hence the data could not be included in this short six-month study.
Responses to the structured questionnaire were the source of data regarding basic infrastructure and specific requirements for “reporting” and “non-reporting” sites (records and observation of basic infrastructure at the institution).
2.4.3. Data Collection
Monthly AMR data reports received from 1 January to 30 June 2019, were assessed for completeness and consistency. On-site AMR surveillance data from the “reporting” sites were checked during visits in January–February 2020. “Non-reporting” sites were also visited during January–February 2020, and the microbiology staff responsible for data compilation were asked to complete the paper-based structured questionnaire. As noted, microbiology staff responsible for data compilation at the “reporting” sites also completed the questionnaire to allow for comparison. The principal investigator also observed the facilities after receiving the questionnaires.
2.5. Data Analysis and Statistics
All positive bacterial culture records from the Excel sheet at the NPHL were checked for consistency with the GLASS criteria (
Supplementary Tables S1 and S2) regarding recording the correct pathogen for the specific specimen type and the specific relevant antibacterial. The specimen–pathogen combination was defined as each priority specimen and its combination of priority pathogen e.g., genital swab and
Neisseria gonorrhoeae or stool specimen and
Salmonella/Shigella spp., and so on. The pathogen–antibacterial combination was defined as each priority pathogen and listed antibiotics to be tested e.g.,
Staphylococcus aureus with cefoxitin and oxacillin. The number and proportion of positive bacterial culture records consistent with the GLASS recommendations were recorded. While checking for completeness of pathogen–antimicrobial combination, the numbers included as total were adjusted according to the antimicrobial sensitivity for each pathogen. Colistin, though mentioned for Gram-negative organisms, was only included in the calculation if second-line antibiotics (e.g., carbapenems) were resistant and carbapenems were included in the calculation only if first-line antibiotics (e.g., third- and fourth-generation cephalosporins) were resistant. The data collected in the Excel sheet at the NPHL were verified with on-site visits to the five “reporting” sites. Before data analysis, the records were checked for duplicates and the ones occurring within a month (repeated isolates of the same bacterial species isolated from a patient within thirty days, regardless of specimen type) were removed. Timeliness was assessed as delay in days. Consistency, completeness, and timeliness were reported separately and cumulatively for the five sites.
The denominator for each site was its total number of checked positive records. The proportion was calculated using the following formula:
The Excel sheet at the NPHL was also verified with records at five reporting surveillance sites (paper-based at four sites and an electronic database at one site where the paper-based record was not available). The paper-based or electronic data at the surveillance sites was taken as the trusted source of data (“gold standard”).
If no discrepancy was found for any variable, a score of one was given for both consistency and completeness. If a discrepancy was found, a score of zero was given for consistency and a score of one for completeness. If there were any missing values in either the Excel sheet from NPHL or the source data, a score of zero was given for both consistency and completeness.
The specific number of variables for each pathogen in the Excel sheet was checked for completeness and consistency by comparison with source data fields. The proportion of data consistency was calculated by adding up the number of consistent data and dividing that value by the total number of expected data elements.
The timeliness was assessed by the timing of reports submitted to the NPHL. If a report was submitted by the 15th working day of the following month, a score of one was given. If it was submitted after the 15th working day, a score of zero was given. The number of days of delay in submitting reports was also calculated between the date of the deadline for each month and the date the report was submitted.
Differences in delays to report submission were assessed as the mean for the study duration, i.e., 6 months. Overall completeness and overall consistency among the five surveillance sites were also assessed.
Box 1 explains the key operational definitions that were used in this study. The names of the surveillance sites were anonymized before any data analysis or publication of the results.
2.6. Ethical Approval
Ethics approval was obtained from the Union Ethics Advisory Group (International Union against Tuberculosis and Lung Disease, Paris, France) EAG No. 64/19 on 21 August 2019, and from the NHRC, Protocol No. ERB 673/2019P, on 29 December 2019. Approval was sought from all relevant stakeholders, including written permission for using the AMR surveillance data from the director of the NPHL and the AMR surveillance site institution/laboratory heads.
Box 1. Operational definitions. 1 AMR, antimicrobial resistance; 2 GLASS, Global Antimicrobial Resistance Surveillance System; 3 NPHL, National Public Health Laboratory.
AMR 1 data | Detailed identification and antibiotic susceptibility data of specific bacterial isolates, along with unique identifiers, specimen, origin, date of sampling, and demographic data from surveillance sites’ microbiology laboratory records. |
Origin | Place: “Hospital” or “Community” origin. |
Timeliness of data | AMR data, for a particular month, received within the 15th working day of the following month. |
Duplicate data | AMR data occurring within a month i.e., repeated isolates of the same bacterial species isolated from a patient within thirty days, regardless of specimen type. |
Specimen–pathogen combination | Combination of priority specimens (namely, blood, urine, stool, or genital swabs) with priority pathogens according to the GLASS2. |
Pathogen–antibacterial combination | Combination of eight priority pathogens and the relevant listed antibiotics according to the GLASS. |
Consistency of data | Data is considered consistent when the variables used to generate the report are as directed by the GLASS manual i.e., four priority specimens and eight priority pathogens. |
Completeness of data | Completeness signifies no missing variables required according to GLASS criteria: age, sex, pathogen, origin, specimen, antibiotic susceptibility results, and date. |
Non-reporting sites | The surveillance sites that have not sent any AMR laboratory data to the NPHL3 for ninety days consecutively. |
Basic infrastructure | Basic facilities and equipment required by the AMR surveillance site to send the AMR reports to the NPHL. |
Specific requirements | Requirements other than the basic infrastructure to send the AMR reports regularly to the NPHL. |
4. Discussion
In this study, we assessed the quality of AMR data sent to the NPHL in Nepal from five of nine investigated AMR surveillance sites. Although considerable in volume, the data lacked completeness and consistency, with the main issues being related to the origin of the isolates (hospital or community setting). The data received at the NPHL also did not follow the GLASS criteria for the specimen–pathogen and pathogen–antibacterial combinations and often were submitted with delay. We looked as well at infrastructure availability at the surveillance sites, and among the four “non-reporting” sites, most lacked dedicated personnel for data entry and had not met some specific requirements, such as having a computer. Other researchers have stressed the need for infrastructure and human resources for reliable data in surveillance [
11].
To track AMR across the globe, the WHO has created the GLASS with the aim of enabling international comparison, analysis, and sharing of AMR data, using information technology. Generation of quality AMR data requires good infrastructure; otherwise, it will be faced with challenges, particularly in resource-limited laboratory settings [
12].
With our study, we have found a lack of uniformity between the stored data at the surveillance sites and the data received by the NPHL. Other researchers have also highlighted a non-uniformity of data that are kept in registers rather than computers [
4,
13]. Schnall et al. identified errors even in the electronic reporting of AMR surveillance data, caused by staffing constraints, which is also a key factor in the lack of reporting uniformity [
12].
In Nepal, Malla et al. [
14] had, by 2014, already highlighted the challenges to AMR surveillance implementation, giving high importance to data collection, compilation, and storage. Many others later noted that the availability of an electronic data collection system with data back-up, as opposed to manual paperwork collection, would result in a smoother workflow and better data flow from the surveillance sites to a central reporting site [
15,
16,
17,
18]. The use of WHONET software, with its uniform and standardized format, would facilitate an easy compilation for GLASS reporting [
19].
Ethiopia has faced similar issues and pledged to develop a better AMR surveillance strategy. However, WHONET software integration for surveillance proved challenging at the data entry site, and they opted for an individualized solution for data entry [
20].
Ibrahim et al. also found data transmission challenges in Ethiopia’s reporting to the GLASS, although they could improve it by on-site mentorship and frequent site visits with weekly calls [
21]. In our study, the AMR surveillance site personnel had received training in how to enter data into WHONET, but two of the four “non-reporting” sites did not use the system because no computer was available.
At most of the surveillance sites, heads of the institution had not signed documents, such as an official memorandum of understanding or a written “Terms of Reference,” recording their commitment to report to the NPHL for the GLASS.
This study is among the first to assess the quality of data received for GLASS reporting, although others have highlighted the advantages of participating in the GLASS [
17].
A limitation of this study is that it was conducted in only one province because of a lack of time and funding. These findings thus likely cannot be generalized to the whole country, given that only half of the surveillance sites could be assessed and only for a short period of time. For instance, one of the two “non-reporting” sites lacking a computer was in the process of receiving one.
Schnall et al. stressed the need for a consistent source of funding for a better surveillance system [
12]. We wish to highlight that such monitoring and evaluation studies could be extensively planned if funding could be arranged and allocated. The way forward would be to expand this study to all of the surveillance sites in the country.