Field Evaluation of the Performance of Two Rapid Diagnostic Tests for Meningitis in Niger and Burkina Faso

New lateral flow tests for the diagnosis of Neisseria meningitidis (Nm) (serogroups A, C, W, X, and Y), MeningoSpeed, and Streptococcus pneumoniae (Sp), PneumoSpeed, developed to support rapid outbreak detection in Africa, have shown good performance under laboratory conditions. We conducted an independent evaluation of both tests under field conditions in Burkina Faso and Niger, in 2018–2019. The tests were performed in the cerebrospinal fluid of suspected meningitis cases from health centers in alert districts and compared to reverse transcription polymerase chain reaction tests performed at national reference laboratories (NRLs). Health staff were interviewed about feasibility. A total of 327 cases were tested at the NRLs, with 26% confirmed Nm (NmC 63% and NmX 37%) and 8% Sp. Sensitivity and specificity were, respectively, 95% (95% CI: 89–99) and 90% (95% CI: 86–94) for Nm and 92% (95% CI: 75–99) and 99% (95% CI: 97–100) for Sp. Positive and negative predictive values were, respectively, 77% (95% CI: 68–85) and 98% (95% CI: 95–100) for Nm and 86% (95% CI: 67–96) and 99% (95% CI: 98–100) for Sp. Concordance showed 82% agreement for Nm and 97% for Sp. Interviewed staff evaluated the tests as easy to use and to interpret and were confident in their readings. Results suggest overall good performance of both tests and potential usefulness in meningitis outbreak detection.


Introduction
Because of its high case fatality (around 10% [1,2]) and epidemic potential, meningococcal meningitis is a major public health threat [3], especially in the "meningitis belt". This region, which stretches from Senegal to Ethiopia, is characterized by a high seasonal incidence of meningococcal meningitis between January and June [4]. Meningococcal meningitis caused by Neisseria meningitidis (Nm) bacteria may cause large outbreaks and six of the 12 Nm serogroups (A, B, C, W, X, and Y) are responsible for invasive forms of meningitis [5]. The epidemiology of Nm meningitis is evolving rapidly, and new variants are continually emerging [6].
A wide choice of meningococcal vaccines covering a variable number of serogroups is currently available [7]. In a context of a diversification of serogroups with epidemic potential in the meningitis belt, it is important to strengthen microbiological surveillance to prompt the early implementation of vaccination campaigns appropriate to circulating serogroups.
Pneumococcal meningitis causes clusters of cases and, less frequently, outbreaks, in the meningitis belt, even following introduction of pneumococcal conjugate vaccines. In 2019, following massive the reduction of NmA, with MenA vaccine rollout, Streptococcus pneumoniae (Sp) was found to be responsible for 40% of meningitis cases in countries in the region [8]. Pneumococcal meningitis has very high case fatality rates (36-66% in the meningitis belt [9]) and, given the difficulty of treatment, requires longer treatment protocols than those for meningococcal meningitis [10].
The microbiological diagnosis of meningitis is currently based on culture or, more frequently, on polymerase chain reaction (PCR) tests carried out by national reference laboratories (NRLs) on a sample of cerebrospinal fluid (CSF) [11]. This last procedure, which requires maintaining the sample in a cold chain in primary health care centers (PHCs) and during transport to the NRL, makes the early identification of circulating strains difficult. To reduce the time needed to detect circulating strains, diagnostic tests have been developed that can be used at bedside and provide a diagnosis in a few minutes. However, currently available rapid diagnostic tests (RDTs) to detect meningococcal meningitis have a short shelf life, are sensitive to heat, and cannot detect all circulating serogroups, making them of little utility in the meningitis belt context [12].
The rapid diagnostic tests (RDT) were first developed at the Institut Pasteur, Paris, as immunochromatographic tests [13,14]. They were thereafter transferred the BioSpeedia company who then developed MeningoSpeed and PneumoSpeed that can be used between 2 and 30 • C to diagnose, respectively, the five main meningococcal serogroups (A, C, W, X and Y) [15] found in the African meningitis belt and Sp meningitis. The tests are based on the use of antibodies directed against capsular polysaccharide of Nm (serogroups A, C, W, Y, and X) and against pneumococcal cell wall polysaccharide C that is common to all Sp isolates. Laboratory based tests suggest good sensitivities and specificities of these two RDTs [12].
These RDTs require the transfer of two drops of CSF in each cassette (three for Nm groups in A/W serogroups, Y/C serogroups and X serogroup; one for Sp meningitis) and a waiting time of 15 min. The reading process is similar to any RDTs, based on control and case lines appearance and they can be performed by all staff after a short demonstration.
The objective of the study was to measure the performance of the MeningoSpeed and PneumoSpeed RDTs in diagnosing Nm and Sp meningitis in CSF at the bedside among patients in PHCs in Niger and Burkina Faso in 2018 and 2019.

Overview
To measure the performance of the RDTs, we compared the results of RDTs conducted at bedside in PHCs with the results of PCRs obtained at NRLs. PHCs in districts where incidence exceeded 3 suspected cases per week per 100,000 inhabitants were invited to participate, and staff were trained, supervised and supplied with RDTs.
To measure the performance of the RDTs after sample transportation, RDTs were also repeated at the NRLs and results compared with PCR tests.
The study population consisted of any consenting patient aged at least two months admitted to a participating PHC with suspected bacterial meningitis (according to the WHO case definition [16]) (sudden onset of fever and stiff neck or other meningeal signs (including bulging fontanelle for patients under 12 months)) between April 2018 and June 2019. Suspected cases with contraindications for lumbar puncture or who refused to participate were excluded from the study.
According to national guidelines for suspected bacterial meningitis case management, medical staff collected CSF through lumbar puncture on every suspected case. For each patient included in the study, the two RDTs (MeningoSpeed and PneumoSpeed) were performed on CSF at the PHC. A 1 mL CSF tube was then refrigerated and sent to the NRL where an additional RDT and a real-time PCR test was performed and used as a gold standard.
PHC staff collected information on patients' symptoms; antibiotic treatment before admission; dates of onset of symptoms, lumbar puncture, RDT reading, and dispatch of sample to the NRL; lot number; and RDT results. PHC information was merged with NRL data, including the results of the PCR tests and the RDTs performed at the NRLs, and data were analyzed using the STATA software package version 14 [17]). The producer of the tests (BioSpeedia Company, Saint Etienne, France) did not fund the trail, did not contribute to the design and did not participate in the analysis of the results. None of the authors is employed by that company.

Performance of the RDTs
MeningoSpeed test results were classified as positive or negative per serogroup A, C, W, Y, or X (positive if positive for that serogroup; negative if negative for that serogroup, regardless of other serogroup's results) and for any Nm (positive if positive for any serogroup A, C, W, Y, or X; negative if negative for all these 5 serogroups). PneumoSpeed test results were classified as positive or negative for Sp meningitis.
We compared the results of the RDTs performed in the PHCs with the results of PCR tests performed at the NRLs to calculate the sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) of the tests. The analyses were stratified by year of lumbar puncture (2018 vs. 2019) and epidemic period (defined as between January and June) versus non-epidemic period.

RDT Feasibility and Acceptability
PHC staff photographed the RDTs as they were reading them. The photographs were then reviewed and the RDTs blindly re-interpreted by independent readers. We compared the results of the RDTs as interpreted by the PHC staff and independent experts and quantified the level of concordance between them using the kappa coefficient [18]. We considered the agreement between the tests moderate if κ < 0.4, average or good if κ = 0.4 to 0.75 and excellent if κ > 0.75. National study coordinators conducted semi-structured interviews with RDT users to assess the acceptability and feasibility of the RDTs.

Sample Size
To estimate an expected 95% sensitivity of MeningoSpeed Nm meningitis identification, with an absolute precision of 5% and a 30% PCR positivity [19], we estimated that at least 243 suspected cases should be included in the study.

Ethical Considerations
The study followed the principles governing biomedical research involving human participation and was carried out in line with principles of the Declaration of Helsinki to ensure that the rights, integrity, and confidentiality of participants were protected. The agreements of the ethics committees of WHO (WHO ERC.0002926), Niger (deliberation N • 35/2017/CNRES) and Burkina Faso (deliberation N • 2017-10-156) were obtained before the start of the study as well as for its extension into 2019.
Written consent was obtained from suspected meningitis cases or from their parents or legal guardians. Standardized information was read to the potential participant. The confidentiality of the data collected, and the anonymity of the participants were ensured during and after the survey (no patient name appears in the databases).
Owing to a lack of tests and a strike by staff, it was only possible to repeat the RDTs at the NRLs in 147 of the 327 patients (45%). Performances of the RDTs at the NRLs and the health centers were comparable (Table 2).

RDT Feasibility and Acceptability
There was an 82% agreement for Nm and 97% for Sp between RDT results registered at the PHCs and photographs reviewed by an independent expert. The kappa coefficient suggested an excellent concordance for Sp and NmX, a good concordance for all Nm and moderate concordance for NmA, NmC, and NmW (Table 5). Eleven NmX, nine NmA, six NmC, two NmW, and one NmY were detected positive with RDTs but were tested negative by real-time PCR.
All nine positive NmA results identified in health centers were from Burkina Faso, eight in 2018 and one in 2019. Of these, two were identified as positive for NmA on photographs by the external expert. The authors confirmed that a faint line, suggestive of a positive NmA result was indeed visible in these two photographs (Figure 1). Of the eleven false positive NmX results, nine were from Burkina Faso and two from Niger. Of those, four photographs were reviewed and classified as negative by the independent experts. One photograph of a false positive NmC result was available and suggested a misreading by the PHC medical staff.
A total of 31 staff were interviewed about their experiences in using the RDTs (20 in Niger and 11 in Burkina Faso). Overall, respondents found the RDTs easy to use (average difficulty score: 1.9/10). They found their interpretation very easy (average difficulty score: 1.2/10) and had great confidence in their result (average lack of confidence score: 1.3/10).
positive NmA result was indeed visible in these two photographs (Figure 1). Of the eleven false positive NmX results, nine were from Burkina Faso and two from Niger. Of those, four photographs were reviewed and classified as negative by the independent experts. One photograph of a false positive NmC result was available and suggested a misreading by the PHC medical staff. A total of 31 staff were interviewed about their experiences in using the RDTs (20 in Niger and 11 in Burkina Faso). Overall, respondents found the RDTs easy to use (average difficulty score: 1.9/10). They found their interpretation very easy (average difficulty score: 1.2/10) and had great confidence in their result (average lack of confidence score: 1.3/10).

Performance of the RDTs
We described in this article the performance results from ready-to-use bedside RDTs, usable by any medical staff, storable at 2-30 °C, allowing to detect Nm and Sp within 15 min. Our results suggest that the RDTs for diagnosing Nm and Sp performed well under field conditions at PHC level, with sensitivity and specificity above 90%. They performed similarly whether used by medical staff at a PHC or technical staff at an NRL. These RDTs were described as acceptable and easy to use in a bedside context. Serogroup-specific Nm RDT performance could be measured for NmC and NmX and were in the same range, above 90%. The two serogroups were the most prevalent in West Africa in 2018-2019 [20,21]. Further studies would be needed to validate their use in detecting NmA, NmW, and NmY due to low prevalence during our study period.
By extending the study over two years we were able to obtain a sample size large enough to measure the performances of the RDTs with acceptable precision. The study protocol was well followed, despite challenges in systematizing the capture of photographs and the use of the RDTs at the NRLs. In addition, recruitment was lower in Burkina Faso due to security issues along with a strike by health workers in 2019. Owing to these limitations, the precision of some secondary outcomes was low. Logistical issues between

Performance of the RDTs
We described in this article the performance results from ready-to-use bedside RDTs, usable by any medical staff, storable at 2-30 • C, allowing to detect Nm and Sp within 15 min. Our results suggest that the RDTs for diagnosing Nm and Sp performed well under field conditions at PHC level, with sensitivity and specificity above 90%. They performed similarly whether used by medical staff at a PHC or technical staff at an NRL. These RDTs were described as acceptable and easy to use in a bedside context. Serogroup-specific Nm RDT performance could be measured for NmC and NmX and were in the same range, above 90%. The two serogroups were the most prevalent in West Africa in 2018-2019 [20,21]. Further studies would be needed to validate their use in detecting NmA, NmW, and NmY due to low prevalence during our study period.
By extending the study over two years we were able to obtain a sample size large enough to measure the performances of the RDTs with acceptable precision. The study protocol was well followed, despite challenges in systematizing the capture of photographs and the use of the RDTs at the NRLs. In addition, recruitment was lower in Burkina Faso due to security issues along with a strike by health workers in 2019. Owing to these limitations, the precision of some secondary outcomes was low. Logistical issues between the PHCs and NRLs, potentially leading to alteration of samples and affecting the measured performance of the RDTs, cannot be excluded. In order to account for such a risk, the protocol included repetition of RDTs at the NRLs. Similar results between RDTs carried out at the PHCs and NRLs suggest that sample alteration was low in this study.
Our results suggest a higher sensitivity of the RDT in detecting NmC in this field context than previously reported in laboratory conditions (95% versus 65%) [15]. This could be due to the fact that the circulating clone in Niger and Burkina Faso (NmC, clonal complex ST-10217) [19,22] is particularly well detected by the antibodies used in the MeningoSpeed test.
PPV for any Nm and NmX were at 77% and 73%, respectively, corresponding to a quarter of false positive RDT readings for these outcomes. Moreover, recurrent false positives for NmA were registered. Further analyses suggested that most of these findings were attributed to specific PHCs and that constant supervision and training of RDT users led to a significant decrease in false positives between the first and the second study years. NmA RDT issues, documented with photographs, were fed back to the manufacturer.
The original study protocol included the use of PneumoSpeed tests on urine samples. However, only 31 tests were reported, precluding any interpretation related to this objective. Considering its potential impact on patients' comfort and practice safety, specific studies should be implemented to measure the performance of RDT tests on urine.
Performance measurements are the results of both the quality of the test and the interpretive capacity of the user. Measurement of the concordance between the interpretations of the nursing staff and the independent expert made it possible to discuss this additional information bias linked to the evaluator. Although the proportion of photographs that could be read was low, concordance was generally good, suggesting a good understanding of the use of the RDTs by the PHC staff.
These field performance results met the WHO target product profile acceptable values for sensitivity and specificity, of >90% [23] and were superior or comparable to field performance values for existing meningococcal rapid tests (69-80% sensitivity and 81-94% specificity for latex agglutination tests and 89-92% sensitivity and 85-99 specificity for lateral flow test) [12]. Our field evaluation confirms the good performance of the tests in laboratory conditions and suggests that the tests are suitable for use in field conditions and that they are acceptable to health personnel, but that they should be accompanied by clear instructions and effective training. Considering this performance, their longer shelf-life and improved thermostability (but still below the desired target product profile value of 40 • ), easier test procedures, and inclusion of all main Nm serogroups (including NmX), MeningoSpeed and PneumoSpeed are good candidate tests for the early detection of meningitis epidemics in Africa. However, this field study is only one step towards ensuring access to safe appropriate diagnostic tests of good quality. At any rate, ensuring confirmation of test results with a more specific test such as PCR will continue to be key, given the decrease in incidence of Nm A and anticipated decrease of other Nm serogroups with future vaccination efforts.
One of the priority goals identified by the Defeating meningitis by 2030 Roadmap is the improvement of diagnosis at all levels of care, through the development and access to diagnostic assays [24]. An expert group gathered by WHO in 2018 [25] identified three essential objectives for the development of in vitro diagnostic tests (IVD) for meningitis diagnosis, including the rapid detection of epidemics in the African meningitis belt. The MeningoSpeed and PneumoSpeed could potentially meet this specific need. Issues that remain to be addressed, before procuring the test more widely, are costs, further thermostability improvements, and scale-up of production capacity with a reliable quality management system. Until elimination of meningitis epidemics in the region is achieved, one of three visionary goals of the roadmap, the use of rapid diagnostic tests will remain an important tool for meningitis control in Africa.