Next Article in Journal
Phylogeography of Puumala orthohantavirus in Europe
Next Article in Special Issue
Characterization of Biomarker Levels in Crimean–Congo Hemorrhagic Fever and Hantavirus Fever with Renal Syndrome
Previous Article in Journal
Viral Long-Term Evolutionary Strategies Favor Stability over Proliferation
Previous Article in Special Issue
Haemostatic Changes in Five Patients Infected with Ebola Virus
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Viruses
Volume 11
Issue 8
10.3390/v11080678
viruses-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Evaluation of Diagnostic Performance of Three Indirect Enzyme-Linked Immunosorbent Assays for the Detection of IgG Antibodies to Ebola Virus in Human Sera

by
Janusz T. Paweska
1,2,*,†,
Naazneen Moolla
1,†,
Nadia Storm
1,†,
Veerle Msimang
1,
Ousman Conteh
3,
Jacqueline Weyer
1 and
Petrus Jansen van Vuren
1
1
Centre for Emerging Zoonotic and Parasitic Diseases, National Institute for Communicable Diseases of the National Health Laboratory Service, Sandringham, Johannesburg 2131, South Africa
2
Faculty of Health Sciences, University of the Witwatersrand, Johannesburg 2050, South Africa
3
Ministry of Health and Sanitation, Freetown 47235, Sierra Leone
*
Author to whom correspondence should be addressed.
†
These authors contributed equally to this work.
Viruses 2019, 11(8), 678; https://doi.org/10.3390/v11080678
Submission received: 29 May 2019 / Revised: 10 July 2019 / Accepted: 11 July 2019 / Published: 24 July 2019
(This article belongs to the Special Issue Medical Advances in Viral Hemorrhagic Fever Research)
Browse Figures
Versions Notes

Abstract

Filovirus serological diagnosis and epidemiological investigations are hampered due to the unavailability of validated immunoassays. Diagnostic performance of three indirect enzyme-linked immunosorbent assays (I-ELISA) was evaluated for the detection of IgG antibody to Ebola virus (EBOV) in human sera. One I-ELISA was based on a whole EBOV antigen (WAg) and two utilized recombinant nucleocapsid (NP) and glycoproteins (GP), respectively. Validation data sets derived from individual sera collected in South Africa (SA), representing an EBOV non-endemic country, and from sera collected during an Ebola disease (EBOD) outbreak in Sierra Leone (SL), were categorized according to the compounded results of the three I-ELISAs and real time reverse-transcription polymerase chain reaction (RT-PCR). At the cut-off values selected at 95% accuracy level by the two-graph receiver operating characteristic analysis, specificity in the SA EBOV negative serum panel (n = 273) ranged from 98.17% (GP ELISA) to 99.27% (WAg ELISA). Diagnostic specificity in the SL EBOV negative panel (n = 676) was 100% by the three ELISAs. The diagnostic sensitivity in 423 RT-PCR confirmed EBOD patients was dependent on the time when the serum was collected after onset of disease. It significantly increased 2 weeks post-onset, reaching 100% sensitivity by WAg and NP and 98.1% by GP I-ELISA.

1. Introduction

High-mortality and occurrence of Ebola disease (EBOD) outbreaks almost each year in the last three decades [1,2] are of great public health concern. The unprecedented and first epidemics of EBOD in West Africa from 2013 to 2016 [3,4,5,6] and the large-scale re-emergence of EBOD in the Democratic Republic of the Congo in 2018 and 2019 [2,7] exemplify the devastating health, humanitarian, and socio-economic impacts and challenges in containing EBOD outbreaks in resource-poor and politically conflicted settings [5,6,7,8,9,10,11,12,13,14].
The increasing incidence, severity and size of EBOD outbreaks highlight the importance of developing and standardizing diagnostic tools for EBOD rapid diagnosis and post-epidemic surveillance. The most devastating EBOD outbreak to date, the West African outbreak, prompted the development of diagnostic assays that can address limited laboratory infrastructure, resources and personnel in affected areas and fulfill requirements for point-of-care (POC) rapid diagnostic tests (RDTs), including real-time reverse-transcription polymerase chain reaction (RT-PCR) based on Cepheid GeneXpert technology [15,16] and antigen detection lateral flow immunoassays [17,18]. While Ebola virus (EBOV) POC and RDTs molecular and virological methods have been successfully developed and evaluated, serological diagnosis, disease serosurveillance and control programs and monitoring of the immune responses in vaccines remain a challenging subject due to unavailability of validated serological assays. Collection of diagnostic specimens after viral clearance and inappropriate transportation methods and storage conditions may negatively impact molecular assays [19,20] making serology testing an important tool in the public health response to outbreaks occurring in remote locations where limited resources are available. Serological tests such as enzyme-linked immunosorbent assays (ELISA) are used for the detection of antibody responses to antigens associated with filovirus infection or vaccination. Establishment of such assays is dependent on the production of high quality immunoreagents [21]. Recombinant antigen technology offers the advantage of development and production under biosafety level two conditions.
Recombinant filoviral nucleoprotein (NP) is an ideal target for ELISAs due to their high immunogenicity and abundance in infected cells [22]. The NPs of filoviruses are highly conserved [23], and ELISAs based on recombinant EBOV NP antigens have been reported to cross-react with sera from humans and animals infected with different filovirus species [24,25,26]. Recombinant NPs are therefore useful for genus-specific screening ELISAs where the specific filovirus species with which a patient is infected is unknown [24,26]. The glycoprotein (GP) is another ideal antigen for filovirus ELISAs as it contains epitopes known to be targets for filovirus-neutralizing antibody responses [27]. Contrary to the NP, the GP is highly diverse [23] and is useful for ELISAs where the main purpose is to detect species-specific filoviral antibodies.
Recombinant protein-based ELISAs for filoviruses were first developed by Prehaud and colleagues in 1998 [28]. The assays made use of either recombinant NP antigens expressed in an Escherichia coli (E. coli) system, or recombinant GP antigens expressed in a baculovirus-Spodoptera frugiperda (Sf9) cell expression system. The antigens reacted positively with sera from previously EBOV-infected human patients [28]. Several other groups have since used and adapted these protein expression techniques for the development of serological assays for EBOV and have reported high sensitivity and specificity for the detection of anti-EBOV IgG [24,29,30,31]. However, recombinant antigens expressed in a bacterial expression system are often problematic due to a lack of molecular folding and post-translational modifications such as glycosylation [32]. Recombinant antigens expressed in baculovirus-insect cell systems prevail over some of the problems presented by those expressed in bacterial expression systems, but the antigenic properties of these antigens may be considerably affected by the differing glycosylation pathways between insect and mammalian cells [33]. To overcome the above-mentioned problems, Nakayama and colleagues [26] developed the first filovirus ELISAs based on recombinant Histidine-tagged GP expressed in human embryonic kidney 293T cells using the mammalian expression vector pCAGGS-MCS [26]. Vu and colleagues developed, optimized, and evaluated a similar ELISA for the quantification of antibodies to EBOV in vaccinated non-human primates, but human samples were unavailable for testing [31]. A field-validated ELISA based on commercially available recombinant EBOV GP has been reported for the detection of immunoglobulin G (IgG) in oral fluid [34]. However, due to the lower sensitivity of oral fluid-based assays compared to serum-based assays, there is a risk of misdiagnosing patients with a low oral concentration of anti-EBOV IgG.
Although a number of new generation filovirus serological assays have been developed in recent years, none have been adequately validated to date for the detection of IgG in sera from EBOV-infected humans. This is largely due to the difficulty in obtaining panels of clinical material sufficient in size to allow for statistically sound evaluation. Nevertheless, the validation of serological assays and determination of their basic diagnostic performance parameters are essential for reporting diagnostic results, comparing patient results between different diagnostic laboratories, evaluating ELISA efficacies, determining seroprevalence rates, infection risk population studies, and assessing occurrence of asymptomatic infections [35].
The purpose of this study was to evaluate and compare the diagnostic performance of EBOV IgG-indirect ELISAs based on antigens produced by classical virological and recombinant protein expression methods using human serum panels from EBOV non-infected and EBOV infected humans.

2. Materials and Methods

2.1. Specimens and Ethics Clearance

A total of 1372 individual banked sera collected in South Africa (SA) and Sierra Leone (SL), were used. SA sera (n = 273) were originally submitted for various routine diagnostic testing to the Centre for Emerging Zoonotic and Parasitic Diseases of the National Institute for Communicable Diseases (NICD), Johannesburg. These sera represented specimens collected from 273 individuals in EBOV non-endemic country and are regarded as IgG EBOV negative reference serum panel. Ethics clearance for using SA human banked sera in the development and validation of diagnostic assays was obtained from Human Ethics Committee, University of the Witwatersrand, Johannesburg, SA, Clearance Certification No. M1809219, 23 February 2018.
SL blood specimens were originally submitted for EBOV RT-PCR testing to the SA modular high-biosafety field Ebola diagnostic laboratory (FEDL) established in August 2014 near Freetown, in international response to the rapidly increasing number of EBOD cases in SL [36]. Selected aliquots of processed sera were shipped on dry ice from FEDL to the NICD’s biosafety level 4 (BSL-4) facility in Johannesburg, SA by the end of March 2015 for long-term storage and post-outbreak investigations. Export permits were obtained from the Pharmacy Board of SL (Nos.: PBSL/061/02 2015, PBSL/061/03 2015). Ethics clearance for using SL human sera in the development and validation of EBOV diagnostic assays was obtained from the Government of Sierra Leone, Office of the Sierra Leone Ethics and Scientific Review Committee, Directorate of Training and Research, Ministry of Health and Sanitation (application version of 12 October 2015, clearance issued on 02 November 2015 and 16 October 2018). The ethics identification codes were not provided on ethics clearances issued by the Office of the Sierra Leone Ethics and Scientific Review Committee.
In total, 1099 sera obtained from SL patients suspected of having EBOD between August 2014 and March 2015 [36] were used. Of those sera 423 were from EBOV RT-PCR confirmed cases for whom date of disease onset was recorded on EBOD case submission form. Ribonucleic acid from blood was extracted using the QIAamp viral RNA kit (Qiagen, Hilden, Germany) according to the manufacturer’s instructions. RT-PCR was performed using the QIAGEN One-Step RT-PCR kit (Qiagen, Hilden, Germany) as per the manufacturer’s instructions using previously described primers and probes targeting the EBOV L gene [37]. Specimens with Ct values below 40 were considered positive for EBOV RNA [36] and thus confirming EBOV infectious status of a patient. Serum specimens from the SL EBOV RT-PCR positive patients were regarded as SL reference positive serum panel.
The remaining specimens were from 676 EBOV RT-PCR negative individuals whose sera tested negative for anti-EBOV IgG by all IgG ELISAs evaluated in this study using cut-offs derived from SA IgG EBOV negative reference serum panel. This serum panel was regarded as SL reference negative serum panel.

2.2. ELISA Serum Controls and Internal Quality Control (IQC)

The source of positive control serum (C++) was the 1996 imported EBOD case from Gabon to SA [38,39]. The case was a Gabonese physician who had been brought from Libreville, Gabon on 27 October and admitted to a private hospital in Johannesburg where a nurse died after being infected through exposure to his blood.
Negative control serum (reference no. CEZPD SVPL 502/2017) was obtained from South African Blood Service. It tested negative for anti-EBOV IgG and IgM antibodies by in-house ELISA using procedures published by Ksiazek et al. [40].
Internal quality control (IQC) data were generated as described previously [41]. Upper and lower control IQC limits together with coefficients of variations ≤10% for replicates of positive internal control serum and test sera were applied as an assay acceptance criteria.

2.3. Enzyme-Linked Immunosorbent Assays

Diagnostic performance of three indirect ELISAs (I-ELISA) for the detection of anti-EBOV IgG antibody in human sera was evaluated. These ELISAs were based on antigens prepared from infected cell lysate, or on recombinant antigens produced in a bacterial or mammalian expression system.

2.3.1. Production of Ebola Virus Whole Antigen

The EBOV whole antigen (WAg) was produced as previously described with some modifications [40]. Vero C1008 cells (ATCC, Manassas, VA, USA) were infected with the SPU 220/96 isolate of EBOV (4th passage in Vero cells) isolated from the serum of a nurse who contracted a fatal infection from a Gabonese physician admitted to a private hospital in South Africa in 1996 [39]. After incubation at 37 °C when cytopathic effect was observed, cells and supernatant were collected and snap freeze-thawed at −70 °C and 37 °C. Lysed cells and supernatant were separated by centrifugation (10,000× g, 4 °C, 15 min). The collected supernatant was gamma irradiated with 30,000 Gy to inactivate the virus. Saturated ammonium sulphate solution (100%) (Sigma Aldrich, Merck, Kenilworth, NJ, USA) was slowly added with constant stirring to the irradiated supernatant to a final concentration of 35%, and the mixture was incubated at 4 °C overnight. The antigen-containing pellet precipitate was collected by centrifugation at 20,000× g for 30 min at 4 °C and resuspended in one tenth of the original supernatant volume of phospate buffered saline (PBS) pH 7.2. The antigen was further dialysed against PBS to remove the ammonium suphate (3–4 buffer changes). The WAg preparation was aliquoted and stored at −70 °C until use. Uninfected Vero C1008 cells were prepared in the same way and used as control antigen.

2.3.2. Prodution of Recombinant Ebola Virus Nucleoprotein

The codon optimized nucleotide sequence for EBOV nucleocapsid (NP) (GenBank accession number AF086833.2) was synthesized with a C-terminal glycine linker to a 6× Histidine tag (GenScript, Piscataway, NJ, USA) and subcloned into the NcoI and XhoI restriction sites of the pET-15b expression vector (Novagen, Merck, USA). The sequence verified plasmid (pET-15b ZEbov NP) was used to transform competent BL21 Star (DE3) E. coli. A starter culture was grown overnight at 37 °C, then diluted 10 fold and allowed to reach exponential-phase growth (OD600 of between 0.6–0.8) before protein expression was induced using 1 mM IPTG (Sigma Aldrich, Merck, USA) for 20 h at 37 °C with vigorous shaking. Cells were harvested by centrifugation, resuspended in sodium phosphate buffer (50 mM NaH2HPO4, 300 mM NaCl, pH 8.0) and lysed using a combination of Bugbuster and Lysonase (Novagen, Merck, USA) treatment, freeze-thaw cycles and sonication. The recombinant NP protein-containing soluble phase was collected by high speed centrifugation (15,000× g, 30 min, 4 °C) and loaded onto Profinity IMAC (Biorad, Hercules, CA, USA) cobalt-charged resin. The protein was left to bind overnight with gentle shaking at 4 °C, using a ratio of 1 mL resin to 16 mL soluble protein fraction. Contaminating proteins were removed by washing the resin twice with 20 packed-resin volumes of sodium phosphate buffer containing 12.5 mM imidazole (pH 8.0), using gentle centrifugation (1000× g, 2 min, 4 °C). The recombinant NP was then eluted overnight with gentle shaking at 4 °C in 2 packed-resin volumes of sodium phosphate buffer containing 250 mM imidazole (pH 8.0). Eluted protein was dialysed into 0.05 M carbonate/bicarbonate buffer, pH 9.6 (Sigma Aldrich, Merck, USA). To remove any residual contaminating proteins, the purified protein was passed through a SEC 650 size exclusion column (Biorad, Hercules, CA, USA) and the fractions containing the NP were collected. Collected fractions were pooled and concentrated using Amicon Ultra filters (Millipore, Merck, USA) with a 50 kD cut-off. The purified NP was quantified using the Bradford Concentration Assay kit (Pierce, ThermoFisher Scientific, Waltham, MA, USA) and aliquots were stored at −70 °C for later use. For the control antigen, the same process was followed using an expression vector (pET-15b) without an insert.

2.3.3. Recombinant Ebola Virus Glycoprotein

Recombinant EBOV glycoprotein (GP) antigen expressed in human embryonic kidney 293 T cells was obtained from Integrated BioTherapeutics (Rockville, MD, USA).

2.3.4. Indirect Enzyme-Linked Immunosorbent Assay Procedures

Optimal immunoreagents concentrations/working dilutions for I-ELISA were determined using standard checkerboard titration procedures [42]. Microtiter 96-wells plates (Maxisorb Immunoplates, Nunc, Roskilde, Denmark) for WAg, NP, and GP I-ELISAs were respectively coated with WAg and corresponding control antigen, NP and corresponding control antigen or with GP. For WAg and NP I-ELISAs virus and control antigens were added to rows of the top half of the plate (rows A–D:1–12) and the bottom half of the plate (rows E–H: 1–12), respectively. For GP I-ELISA all rows (rows A–H: 1–12) were coated with the commercial GP for which control antigen was not available. The NP and corresponding control antigen were diluted 1:2000 (stock concentration 0.4 mg/mL) in carbonate–bicarbonate buffer, pH 9.6. All the other antigens were diluted in phosphate PBS without Mg and Ca, pH 7.4; WAg and uninfected Vero C1008 cell culture antigen was diluted 1:300 and the GP was diluted 1:6000 (stock concentration 3.83 mg/mL).
Each 96-well plate had four replicates of positive serum control (C++), two replicates of negative serum control (C−) and two replicates of conjugate control (Cc). For WAg and NP I-ELISA C++ was added to wells A 1–2 and B 1–2 coated with virus antigen and to corresponding wells E 1–2 and F 1–2 coated with a control antigen. Accordingly, C− was added to C 1–2 and G 1–2, and Cc (diluent buffer) was added to D 1–2 and H 1–2. The remaining wells (A–D 3–12 and E–H 3–12) were used for testing 20 test sera in duplicate with test serum no. 1 added to wells A3 and B3 and corresponding E3 and F3; adding of the remaining test sera followed the same layout principle. For GP I-ELISA, not having control antigen, internal controls were placed as follows: C++ in wells A 1–2 and B 1–2; C− in wells C 1–2; Cc in wells D 1–2. The remaining wells (A–D 3–12 and E–H 1–12) were used for testing 44 test sera in duplicate with test serum no. 1 added to wells A3 and B3 and accordingly test serum no. 44 added to wells G12 and H12.
All reagents were added to the immunoplates at a volume of 100 µL/well unless otherwise stated. Passive adsorption onto ELISA plates was performed at 4 °C overnight and all subsequent incubations (except for substrate addition) were performed at 37 °C in a humidified chamber for 1 h. Following coating, plates were washed 3 times with 300 µL of PBS containing 0.1% Tween 20; the same washing procedure followed each subsequent stage of I-ELISAs. Plates were blocked with 200 µL 10% non-fat milk powder in PBS. After incubation, plates were washed, and control and test sera diluted 1:400 in 2% non-fat milk powder in PBS (diluent buffer) were added. Each test serum, negative control serum and conjugate control was tested in duplicate, and positive control was tested in quadruplicate. Following incubation with the sera, the plates were washed and goat anti-human IgG HRPO conjugate (Invitrogen, ThermoFisher Scientific, USA), diluted 1:10,000 in diluent buffer, was added to the wells. After incubation, plates were washed and 2,2’-azinodiethylbenzothiazoline sulfonic acid (ABTS) peroxidase (Seracare Lifesciences, Milford, MA, USA) substrate added to wells. Plates were incubated in the dark at room temperature (±22 °C) for 30 min.
Reactions were stopped by the addition of 1% sodium dodecyl sulphate (SDS) and the optical densities (OD) readings were measured at 405 nm. OD readings were converted into a percentage of the positive internal control serum (PP) using the equation as previously described [41]. Briefly, a specific activity of each serum (net OD) was calculated by subtracting the non-specific background OD in the wells with control antigen from the OD in wells with virus antigen (for NP and WAg I-ELISA only). The mean net OD readings were converted to PP using the formula: PP = (mean net of test serum/mean net OD of positive control) × 100 [41].

2.4. Selection of Cut-off Values and Statistical Analysis

Cut-off values were determined as mean +3 standard deviations (SD) of PP values recorded in SA and SL IgG EBOV negative serum panels and also selected at 95% accuracy level by the misclassification cost term (MCT) option of the two-graph receiver operating characteristics (TG-ROC) analysis available from Microsoft Excel 2010 (Redmond, WA, USA) [43,44] using SL reference IgG EBOV negative and positive serum panels. Coefficient of variation (CV) values were calculated to measure relative variability using the formula CV = (SD of replicates/mean of replicates) × 100.
Optimisation of cut-off values by MCT TG-ROC was based on the following equation: MCT = (1-p) (1-Sp) + rp (1-Se), where p (prevalence) = 0.5 and r (cost of false-positive and false-negative results) = 1.0. I-ELISAs diagnostic sensitivity (DSe) and specificity (DSp) were calculated as follows [45]: DSe = [TP/(TP + FN)] × 100; DSp = [TN/(TN + FP)] × 100; where TP = true positive sera (true EBOD patients), FN = false-negative sera (confirmed infected EBOD patients that tested negative), TN = true negative sera (individuals that truly are negative for EBOD), FP = false-positive (individuals negative for EBOD that test positive). Using Stata 13 (StataCorp, College Station, TX, USA), we obtained DSe and DSp statistics with their 95% confidence intervals for each of the diagnostic tests.
To compare the diagnostic performance of the assays and agreements between results of matched test samples, data were analyzed in Stata 13 using McNemar’s test and Cohen’s kappa statistic (κ) [46].

2.5. Serum Inactivation

To evaluate the effect of serum inactivation on the levels of the detectable anti-EBOV IgG by WAg, NP and GP I-ELISAs, laboratory protocols previously shown to completely inactivate EBOV virus in human serum were used [47,48]. Briefly, the C++ was first diluted 1:10 in PBS containing either 0.5% Triton X-100 or 0.5% Tween-20 (Sigma-Aldrich, Taufkichen, Germany) and heated at 60 °C for 15 min. Then, two-fold log10 dilutions (from 1:100 to 1:200,480; log10102−log10105.3) of untreated and treated serum was tested by each of the I-ELISAs. Each dilution was tested in duplicate on 3 separate runs. A serum titer was considered the highest sample dilution at which its PP value was ≥ the I-ELISA cut-off for at least 50% of six replicates.

3. Results

3.1. IQC and Repeatability

Both within and between the runs, the C++ OD readings for WAg, NP and GP I-ELISAs were within the IQC lower (LCL) and upper (UCL) control limits. Within the runs, the average CV ranged between 3.54 ± 1.99 (NP I-ELISA) and 4.76 ± 2.16 (GP I-ELISA). Between the runs, the average CV ranged between 3.11 ± 1.05 (NP I-ELISA) and 4.8 ± 1.56 (GP I-ELISA) (Table 1). Both within and between the runs, the C− and Cc OD readings were within the IQC LCL and UCL. For WAg, NP, and GP I-ELISA, the IQC C− OD control limits ranged from −0.08 to 0.06, −0.08 to 0.09, 0.1 to 0.17, and the IQC Cc− OD control limits ranged from −0.05 to 0.06, −0.05 to 0.04, and 0.03 to 0.11, respectively.

3.2. Distribution of I-ELISA PP Values in EBOV IgG Negative Sera and the Selection of Cut-off Values

The distribution of I-ELISA PP values and determination of cut-offs as mean plus three standard deviations of PP values recorded by each test in SA and SL EBOV IgG negative serum panels are shown in Figure 1. Selection and optimization of cut-offs by the MCT TG-ROC in SL serum panels is shown in Figure 2.
MCT TR-ROC was based on the non-parametric program option due to departure from a normal distribution of data sets analyzed. PP threshold values for each of the assays were similar irrespective of serum panels analyzed and the methods used for the determination of cut-offs: 13.46, 13.12 (Figure 1A) and 13.35 (Figure 2A) for WAg I-ELISA; 16.13, 15.15 (Figure 1B) and 16.44 (Figure 2B) for NP I-ELISA; 26.52, 25.12 (Figure 1C) and 26.28 (Figure 2C) for GP I-ELISA, respectively. The higher cut-off values for GP I-ELISA were likely due to higher ELISA noise resulting from not including a control antigen in this assay.

3.3. Comparison of D-Sp and D-Se

Irrespective of data set analyzed and cut-offs used, all the three assays evaluated in this study had high estimates of D-Sp, ranging from 97.4% to 99.6% in SA, and from 99.7% to 100% in SL negative serum panels. However, D-Sp for GP I-ELISA was generally lower in the SA negative serum panel (Table 2).
Individual results yielded by WAg, NP and GP I-ELISAs in sera from 423 RT-PCR confirmed EBOD cases at different times post disease onset and using different cut-off values are given in Table 3. Irrespective of the cut-offs used, the results were similar for the same assay, but the NP I-ELISA was more sensitive in detecting IgG antibody during the first two weeks post disease onset compared to WAg and GP I-ELISAs, the latter being the least sensitive. For example, when using the TG-ROC derived threshold, of 335 EBOD patients bled during the two weeks post onset, 92 (27.5%), 114 (34%), and 75 (22.4%) were positive by WAg, NP, and GP I-ELISA, respectively. Mc Nemar test indicate a disagreement of diagnostic capacity (combined DSe and DSp) between NP and WAg (p = 0.0196) or GP I-ELISA (p = 0.0002) respectively using all Sierra Leone data (n = 1099). Agreement was found between WAg and GP I-ELISA (p = 0.0707; 98.18% κ = 0.9300 ± 0.0348). Most of the discrepant (non-matching) results were recorded during the first two weeks post disease onset. After two weeks post onset, detection of IgG antibody and agreement between assays significantly improved. Of 88 sera tested 15–31 days post onset, all tested positive by WAg and NP I-ELISs irrespective of the cut-off used, and depending on the cut-off, 87 or 86 were positive by GP I-ELISA (Table 3). Mc Nemar test indicate agreement between NP, Wag and GP I-ELISAs ranging from 99.35% (κ = 0.9684 ± 0.0362) to 100% using data from non-EBOD and diseased patients bled on day 15 or later post-onset (n = 764), except for GP and Wag or NP I-ELISAs with the cut-offs of respectively 26.28, 13.53, 16.44 (p < 0.001).
Estimates of D-Se in sera collected at different times post disease onset are given in Table 4. Between 8–36 days post onset, the D-Se ranged from 82.6% (GP I-ELISA) to 92.65 (NP I-ELISA). Between 15–36 days post onset, the D-Se was 100% for both WAg and NP I-ELISA irrespective of the cut-off used, and ranged from 97.7% to 98.9% for the GP I-ELISA.
Mean levels of IgG responses measured by WAg, NP, and GP I-ELISAs in 423 EBOD RT-PCR confirmed cases bled at different times after disease onset are shown in Figure 3. On average, the first seroconversions were detectable by NP on days 2–4, then on days 5–7 by NP and WAg, and from day 8 post onset by all I-ELISAs. The mean levels of IgG measured by all the I-ELISAs were not significantly different.
The different inactivation protocols used did not have an adverse effect on the kinetics and the detectable levels of the anti-EBOV IgG in C++ by either I-ELISA evaluated in this study. The titers (Table 5) as well as the kinetics of dose-response curves (Figure 4) were similar before and after inactivation in each assay.

4. Discussion

The diagnostic decision limit or cut-off represents a serological assay test value used to dichotomize negative and positive results, and by inference, to define the infection status of an individual against a specific pathogen of disease. The relevance of data used for the determination of cut-off consequently impacts on estimates of D-Se and D-Sp and other measures of test performance [49]. Important consideration in determining a serological assay cut-off is to select sera from unequivocally infected individuals and sera from individuals who have never been infected with the agent in question. Also, in order to account for the distribution of covariate factors (genetic, nutritional, geographical, and stage of infection) that may influence the estimates of D-Se and D-Sp, the target population should preferably be sampled using simple random, systematic or stratified sampling methods [45]. These ideal conditions could not be applied during this study. An assay validation data should preferably be derived not only from testing samples from reference individuals of known history and infection status but also from the country or region in which the test is to be used. Traditionally, gold standards for selection of truly infected and uninfected subjects include isolation of the agent or pathognomonic histopathological criteria. Because a true gold standard is difficult to accomplish, relative standards of comparison are often necessary, and include results from other serological assays [50]. In order to ensure EBOV true infection status of individuals whose sera were used in our study, RT-PCR negative sera that tested negative for anti-EBOV IgG by all IgG ELISAs using cut-offs derived from SA IgG EBOV negative reference serum panel were regarded as SL reference negative serum panel. Serum specimens from the SL EBOV RT-PCR positive patients with known dates of disease onset were regarded as SL reference positive serum panel.
Various statistical analyses used in our study for the selection of the cut-off values provided similar results. A cut-off value determined as two or three SD above the mean in uninfected individuals is frequently used for the interpretation of serodiagnostic tests. However, this assumes a normal distribution of the test values in population targeted by an assay, and provides only an estimate of D-Sp [49]. Deviations from normality are often recorded in serological data and should be addressed in the selection of threshold values [51]. Therefore, we also used the TG-ROC analysis for the selection and optimization of cut-off values to account for parametric versus nonparametric distribution of test values. All I-ELISAs evaluated in our study had high estimates of D-Sp, but the D-Se was dependent on the time when the serum was taken post disease onset. High detection rate of anti-EBOV IgG in RT-PCR EBOV confirmed cases was only recorded after two weeks post disease onset. Results of previous study in a small number of the 1995 EBOD patients in Kikwit, Democratic Republic of Congo, suggested that many patients do not have antibody early in the course of their illness, and that many may die without developing detectable antibodies to EBOV. Therefore, measurement of IgG antibody is of rather limited use in the diagnosis of acute EBOD cases [52].
I-ELISA, represents one of the simplest ELISA formats, but can be difficult to validate because of signal amplification of both specific and non-specific components [42]. For these reasons, to determine the specific binding of antibody, sera should be tested with both a specific viral antigen and its corresponding control or comparison antigen to account for possible non-specific background activity. The GP I-ELISA evaluated in our study was based on a commercially available recombinant GP for which a negative control could not be obtained. Compared to WAg and NP I-ELISAs, which included control antigens, the higher GP I-ELISA test values in EBOV negative serum panels and consequently the higher PP cut-off values derived for this assay are likely due to not having a control antigen. Due to inherent differences amongst assay systems, binding-antibody levels should be expressed in relative rather than absolute terms. One of the advantages of using PP values as a measure of antibody activity in the I-ELISA is that this method of OD readings conversion does not assume a uniform background activity, and therefore it is also more suitable for inter-laboratory standardization [53].
The first ELISAs for the detection of antibodies to EBOV were based on whole antigen prepared from infected cell lysate [40], and variations of this assay are still widely used in diagnostic and research laboratories [3]. While whole filovirus antigens can be relatively easily produced in large volumes, their preparation pose health risks, restricting their production to biosafety level 4 (BSL-4) facilities. These facilities are not only very expensive to construct and operate, but also are not easily available or accessible for countries where fatal filoviruses are endemic. In addition, the binding of antibodies to cellular contaminants present in EBOV-infected cell lysates may lead to cross-reactivity, resulting in reduced specificity [54]. High quality filovirus recombinant protein antigens can be safely prepared without the need for high BSL-4 biocontainment facilities and outside EBOV endemic areas. Their use in ELISA has potential to reduce the risk of false positive results and allows for better standardization [55,56]. Testing clinical specimens potentially containing a BSL-4 viral agent presents a serious biohazard. While a number of inactivation methods were shown to completely inactivate EBOV [47,48], they also markedly alter the protein components in human blood, e.g., enzymes and coagulation factors [47]. Results of viral inactivation protocols evaluated in our study indicate that they do not alter detectable levels of anti EBOV-IgG, thus together with recombinant antigen based ELISAs, provide a safe and reliable testing platform.
It has previously been reported that the use of single filoviral proteins as antigens is disadvantageous for filovirus serology [28,29,55]. Sera from patients infected with EBOV contain antibodies to several viral proteins and therefore might display reduced activity or later seroconversion in ELISAs based on a single recombinant antigen [55]. Results of our study indicate that anti-EBOV NP IgG is detectable earlier then anti-EBOV GP IgG antibody during the first week post disease onset. This is likely due to higher abundance of this highly immunogenic protein in infected cells [24].
Long-lasting persistence of IgG antibody in humans after infection with EBOV [40,57,58], renders IgG detection ELISA a suitable tool for epidemiological investigations.
Evaluation of the efficacy of filovirus vaccines and therapeutics require monitoring of immune responses that correlate with protection and survival using reliable and reproducible serological methods. Anti-EBOV GP IgG ELISA was recently shown to reproducibly quantify levels of anti-EBOV IgG antibodies in sera from EBOD survivors and immunized individuals [59], thus offering an important laboratory tool for assessing immunogenicity of candidate EBOV vaccines. The NP IgG I-ELISA evaluated in this study has a potential to be used for testing immunity and evaluating protection against natural EBOV exposure in individuals immunized with EBOV GP-based vaccines or receiving anti-EBOV GP passive immunization.

5. Conclusions

As highly accurate, robust and safe tests, the ELISAs based on recombinant EBOV antigens have a potential to replace traditional serological diagnostic methods which pose health risks thus necessitating their use only in high biocontainment facilities. Practically simple viral inactivation protocols evaluated in our study do not alter measurable levels of anti EBOV-IgG, thus together with recombinant antigen based ELISAs provide a safe testing application. Long-lasting IgG antibody makes their detection useful for epidemiological investigations.

Author Contributions

J.T.P. conceptualized the study and provided supervision, performed experiments, analyses and interpretation, provided main funding and wrote the manuscript; N.M. co-conceptualized the study, performed experiments, and co-wrote the manuscript; N.S. performed experiments, co-wrote the manuscript; V.M. performed statistical analyses; O.C. performed experiments; J.W. performed experiments, provided co-supervision; P.J.v.V. co-conceptualized the study, provided co-supervision, performed experiments and interpretation, co-wrote the manuscript. All authors reviewed the manuscript.

Funding

This research was funded by the South African Medical Research Council (Grant no. 011-93254) with funds received from the South African National Department of Health.

Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

References

  1. Kuhn, J.H.; Adachi, T.; Adhikari, N.K.J.; Arribas, J.R.; Bah, I.E.; Bausch, D.G.; Bhadelia, N.; Borchert, M.; Broch Brantsaeter, A.; Brett-Major, D.M.; et al. New filovirus disease classification and nomenclature. Nat. Rev. Microbiol. 2019, 17, 261–263. [Google Scholar] [CrossRef] [PubMed]
  2. Muyembe-Tamfum, J.J.; Mulangu, S.; Masumu, J.; Kayembe, J.M.; Kemp, A.; Paweska, J.T. Ebola virus outbreaks in Africa: Past and present. Onderstepoort J. Vet. Res. 2012, 79, 451. [Google Scholar] [CrossRef] [PubMed]
  3. Baize, S.; Pannetier, D.; Oestereich, L.; Rieger, T.; Koivogui, L.; Magassouba, N.; Soropogui, B.; Sow, M.S.; Keita, S.; de Clerck, H.; et al. Emergence of Zaire Ebola virus disease in Guinea. N. Engl. J. Med. 2014, 371, 1418–1425. [Google Scholar] [CrossRef] [PubMed]
  4. Briand, S.; Bertherat, E.; Cox, P.; Formenty, P.; Kieny, M.P.; Myhre, J.K.; Roth, C.; Shindo, N.; Dye, C. The international Ebola emergency. N. Engl. J. Med. 2014, 371, 1180–1183. [Google Scholar] [CrossRef] [PubMed]
  5. WHO Ebola Response Team. Ebola virus disease in West Africa—the first 9 months of the epidemic and forward projections. N. Engl. J. Med. 2014, 371, 1481–1495. [Google Scholar] [CrossRef]
  6. World Health Organization—Ebola virus Disease—Democratic Republic of the Congo. Available online: https://www.who.int/csr/don/07-february-2019-ebola-drc/en/ (accessed on 11 February 2019).
  7. Médecins Sans Frontières. DRC Ebola outbreak crisis update—April 2019. Available online: https://www.msf.org/drc-2018-ebola-outbreak-crisis-update (accessed on 18 April 2018).
  8. Epstein, H. 11 313 unnecessary Ebola-related deaths: Building citizen trust in health systems. Brenthurst Foundation Special Report 1/2016. Sheaf Publishing: Benoni, South Africa. Available online: www.thebrenthurstfoundation.org/Brenthurst-special-report-201601.p (accessed on 20 May 2019).
  9. World Bank Group. The economic impact of the 2014 Ebola epidemic: Short- and medium-term estimates for West Africa. Available online: https://openknowlege.worldbank.org/handle/10986/20592. (accessed on 20 May 2019).
  10. Evans, D.K.; Goldstein, M.; Popova, A. Health-care worker mortality and the legacy of the Ebola epidemic. Lancet Glob. Health 2015, 3, e439–e440. [Google Scholar] [CrossRef]
  11. Gostin, L.O.; Fridman, A.A. A retrospective and prospective analysis of the West African Ebola virus disease epidemic: Robust national health systems at the foundation and an empowered WHO at the apex. Lancet 2015, 385, 1902–1909. [Google Scholar] [CrossRef]
  12. Shoman, H.; Karafillakis, E.; Rawaf, S. The link between the West African Ebola outbreak and health systems in Guinea, Liberia and Sierra Leone: A systematic review. Glob. Health 2017, 13, 1. [Google Scholar] [CrossRef]
  13. Vinck, P.; Pham, P.N.; Bindu, K.K.; Bedford, J.; Nilles, E. Institutional trust and misinformation in the response to the 2018-19 Ebola outbreak in North Kivu, DR Congo; a population-based survey. Lancet Infect. Dis. 2019, 19, 529–536. [Google Scholar] [CrossRef]
  14. Gostin, L.; Phelan, A.; Coutinho, A.G.; Eccleston-Turner, M.; Erondu, N.; Filani, O.; Inglesby, T.; Katz, R.; Maleche, A.; Nuzzo, B.; et al. Ebola in the Democratic Republic of the Congo; time to sound a global alert? Lancet Infect. Dis. 2019, 393, 617–620. [Google Scholar] [CrossRef]
  15. Jansen van Vuren, P.; Grobbelaar, A.; Storm, N.; Conteh, O.; Konneh, K.; Kamara, A.; Sanne, I.; Paweska, J.T. Comparative evaluation of the diagnostic performance of the prototype Cepheid GeneXpert Ebola assay. J. Clin. Microbiol. 2016, 54, 359–367. [Google Scholar] [CrossRef] [PubMed]
  16. Raftery, P.; Condell, O.; Wasunna, C.; Kpaka, J.; Zwizwai, R.; Nuha, M.; Fallah, M.; Freemen, M.; Harris, V.; Miller, M.; et al. Establishing Ebola virus disease (EVD) diagnostics using GeneXpert technology at a mobile laboratory in Liberia: Impact on outbreak response, case management and laboratory systems strengthening. PLoS Negl. Trop. Dis. 2018, 12, e0006135. [Google Scholar] [CrossRef] [PubMed]
  17. Phan, J.C.; Pettitt, J.; George, J.S.; Fakoli, L.S., III; Taweh, F.M.; Bateman, S.L.; Bennett, R.S.; Norris, S.L.; Spinnler, D.A.; Pimentel, G.; et al. Lateral flow immunoassays for Ebola virus disease detection in Liberia. J. Infect. Dis. 2016, 214, S222–S228. [Google Scholar] [CrossRef] [PubMed]
  18. Wonderly, B.; Jones, S.; Gatton, M.L.; Barber, J.; Killip, M.; Hudson, C.; Carter, L.; Sipson, A.J.H.; Urassa, W.; Chua, A.; et al. Comparative performance of four rapid Ebola antigen-detection lateral flow immunoassays during the 2014-2016 Ebola epidemic in West Africa. PLoS ONE 2019, 7, e0212113. [Google Scholar] [CrossRef] [PubMed]
  19. MacNeil, A.; Reed, Z.D.; Rollin, P.E. Serologic cross-reactivity of human IgM and IgG antibodies to five species of Ebola virus. PLoS Negl. Trop. Dis. 2011, 5, e1175. [Google Scholar] [CrossRef] [PubMed]
  20. Liu, Y.; Sun, Y.; Wu, W.; Li, A.; Yang, X.; Zhang, S.; Li, C.; Su, Q.; Cai, S.; Sun, D.; et al. Serological investigation of laboratory-confirmed and suspected Ebola virus disease patients during the late phase of the Ebola outbreak in Sierra Leone. Virol. Sin. 2018, 33, 323–334. [Google Scholar] [CrossRef] [PubMed]
  21. Cross, R.W.; Ksiazek, T.G. Production of antigens for ELISA. In Ebolaviruses. Methods in Molecular Biology; Hoenen, T., Groseth, A., Eds.; Humana Press: New York, NY, USA, 2017; Volume 1628, pp. 353–362. [Google Scholar]
  22. Niikura, M.; Ikegami, T.; Saijo, M.; Kurata, T.; Kurane, I.; Morikawa, S. Analysis of linear B-cell epitopes of the nucleoprotein of Ebola virus that distinguish Ebola virus subtypes. Clin. Diagn. Lab. Immunol. 2003, 10, 83–87. [Google Scholar] [CrossRef]
  23. Sanchez, A.; Kiley, M.P.; Holloway, B.P.; Auperin, D.D. Sequence analysis of the Ebola virus genome: Organization, genetic elements, and comparison with the genome of Marburg virus. Virus Res. 1993, 29, 215–240. [Google Scholar] [CrossRef]
  24. Saijo, M.; Niikura, M.; Morikawa, S.; Ksiazek, T.G.; Meyer, R.F.; Peters, C.J.; Kurane, I. Enzyme-linked immunosorbent assays for detection of antibodies to Ebola and Marburg viruses using recombinant nucleoproteins. J. Clin. Microbiol. 2001, 39, 1–7. [Google Scholar] [CrossRef]
  25. Natesan, M.; Jensen, S.M.; Keasey, S.L.; Kamata, T.; Kuehne, I.E.; Stonier, S.W.; Lutwama, J.J.; Lobel, L.; Dye, J.M.; Ulrich, R.G. Human survivors of disease outbreaks caused by Ebola or Marburg virus exhibit cross-reactive and long-lived antibody responses. Clin. Vaccine Immunol. 2016, 23, 717–724. [Google Scholar] [CrossRef]
  26. Nakayama, E.; Yokoyama, A.; Miyamoto, H.; Igarashi, M.; Kishida, N.; Matsuno, K.; Marzi, A.; Feldmann, H.; Ito, K.; Saijo, M.; et al. Enzyme-linked immunosorbent assay for detection of filovirus species-specific antibodies. Clin. Vaccine Immunol. 2010, 17, 1723–1728. [Google Scholar] [CrossRef] [PubMed]
  27. Takada, A.; Ebihara, H.; Jones, S.; Feldmann, H.; Kawaoka, Y. Protective efficacy of neutralizing antibodies against Ebola virus infection. Vaccine 2007, 25, 993–999. [Google Scholar] [CrossRef] [PubMed]
  28. Prehaud, C.; Hellebrand, E.; Coudrier, D.; Volchkov, V.E.; Volchkova, V.A.; Feldmann, H.; Le Guenno, B.; Bouloy, M. Recombinant Ebola virus nucleoprotein and glycoprotein (Gabon 94 strain) provide new tools for the detection of human infections. J. Gen. Virol. 1998, 79, 2565–2572. [Google Scholar] [CrossRef] [PubMed]
  29. Groen, J.; van den Hoogen, B.G.; Burghoorn-Maas, C.P.; Fooks, A.R.; Burton, J.; Clegg, C.J.; Zeller, H.; Osterhaus, A.D. Serological reactivity of baculovirus-expressed Ebola virus VP35 and nucleoproteins. Microbes Infect. 2003, 5, 379–385. [Google Scholar] [CrossRef]
  30. Huang, Y.; Zhu, Y.; Yang, M.; Zhang, Z.; Song, D.; Yuan, Z. Nucleoprotein-based indirect enzyme-linked immunosorbent assay (indirect ELISA) for detecting antibodies specific to Ebola virus and Marburg virus. Virol. Sin. 2014, 29, 372–380. [Google Scholar] [CrossRef] [PubMed]
  31. Vu, H.; Shulenin, S.; Grolla, A.; Audet, J.; He, S.; Kobinger, G.; Unfer, R.C.; Warfield, K.L.; Aman, M.J.; Holtsberg, F.W. Quantitative serology assays for determination of antibody responses to Ebola virus glycoprotein and matrix protein in nonhuman primates and humans. Antiviral Res. 2016, 126, 55–61. [Google Scholar] [CrossRef] [PubMed]
  32. Rosano, G.L.; Ceccarelli, E.A. Recombinant protein expression in Escherichia coli: Advances and challenges. Front. Microbiol. 2014, 5, 172. [Google Scholar] [CrossRef]
  33. Jarvis, D.L.; Kawar, Z.S.; Hollister, J.R. Engineering N-glycosylation pathways in the baculovirus-insect cell system. Curr. Opin. Biotechnol. 1998, 9, 528–533. [Google Scholar] [CrossRef]
  34. Glynn, J.R.; Bower, H.; Johnson, S.; Houlihan, C.F.; Montesano, C.; Scott, J.T.; Semple, M.G.; Bangura, M.S.; Kamara, A.J.; Kamara, O.; et al. Asymptomatic infection and unrecognised Ebola virus disease in Ebola-affected households in Sierra Leone: A cross-sectional study using a new non-invasive assay for antibodies to Ebola virus. Lancet Infect. Dis. 2017, 17, 645–653. [Google Scholar] [CrossRef]
  35. Shurtleff, A.C.; Biggins, J.E.; Keeney, A.E.; Zumbrun, E.E.; Bloomfield, H.A.; Kuehne, A.; Audet, J.L.; Alfson, K.J.; Griffiths, A.; Olinger, G.G.; et al. Standardization of the filovirus plaque assay for use in preclinical studies. Viruses 2012, 4, 3511–3530. [Google Scholar] [CrossRef]
  36. Paweska, J.T.; Jansen van Vuren, P.; Meier, G.H.; Le Roux, C.; Conteh, O.S.; Kemp, A.; Fourie, C.; Naidoo, P.; Naicker, S.; Ohaebosim, P.; et al. South African Ebola diagnostic response in Sierra Leone: A modular high biosafety field laboratory. PLoS Negl. Trop. Dis. 2017, 11, e0005665. [Google Scholar] [CrossRef] [PubMed]
  37. Panning, M.; Laue, T.; Olschlager, S.; Eickmann, M.; Becker, S.; Raith, S.; Georges Courbot, M.C.; Nilsson, M.; Gopal, R.; Lundkvist, A.; et al. Diagnostic reverse-transcription polymerase chain reaction kit for filoviruses based on the strain collections of all European biosafety level 4 laboratories. J. Infect. Dis. 2007, 196, S199–S204. [Google Scholar] [CrossRef] [PubMed]
  38. World Health Organization. Ebola hemorrhagic fever—South Africa. Wkly. Epidemiol. Rec. 1996, 71, 359. [Google Scholar]
  39. Richards, G.A.; Murphy, S.; Jobson, R.; Mer, M.; Zinman, C.; Taylor, R.; Swanepoel, R.; Duse, A.; Sharp, G.; de la Rey, I.C.J.; et al. Unexpected Ebola virus in a tertiary setting: Clinical and epidemiological aspects. Crit. Care Med. 2000, 28, 240–244. [Google Scholar] [CrossRef] [PubMed]
  40. Ksiazek, T.G.; West, C.P.; Rollin, P.E.; Jahrling, P.B.; Peters, C.J. ELISA for the detection of antibodies to Ebola viruses. J. Infect. Dis. 1999, 179, S192–S198. [Google Scholar] [CrossRef]
  41. Paweska, J.T.; Burt, F.J.; Swanepoel, R. Validation of IgG-sandwich and IgM-capture ELISA for the detection of antibody to Rift Valley fever virus in humans. J. Virol. Methods 2005, 124, 173–181. [Google Scholar] [CrossRef]
  42. Crowther, R. Elisa Theory and Practice, in Methods in Molecular Biology; Crowther, R., Ed.; Humana Press Inc.: Tatowa, NJ, USA, 1995. [Google Scholar]
  43. Greiner, M. Two-graph receiver operating characteristics (TG-ROC)—A Microsoft-Excel template for the selection of cut-off values in diagnostic tests. J. Immunol. Methods 1995, 185, 123–132. [Google Scholar] [CrossRef]
  44. Greiner, M. Two-graph receiver operating characteristics (TG-ROC)—A Microsoft-Excel template for the selection of cut-off values that minimize overall misclassification costs. J. Immunol. Methods 1996, 191, 93–94. [Google Scholar] [CrossRef]
  45. Greiner, M.; Gardner, I.A. Application of diagnostic tests in veterinary epidemiological studies. Prev. Vet. Med. 2000, 45, 43–59. [Google Scholar] [CrossRef]
  46. Landis, J.R.; Koch, G.G. The measurement of observer agreement for categorical data. Biometrics 1977, 33, 159–174. [Google Scholar] [CrossRef]
  47. Mitchell, S.W.; McCormick, J.B. Physicochemical inactivation of Lassa, Ebola and Marburg viruses and effect on clinical laboratory analyses. J. Clin. Microbiol. 1984, 20, 486–489. [Google Scholar] [PubMed]
  48. Haddock, E.; Feldmann, F.; Feldmann, H. Effective chemical inactivation of Ebola virus. Emerg. Infect. Dis. 2016, 22, 1292–1294. [Google Scholar] [CrossRef] [PubMed]
  49. Jacobson, R.H. Validation of serological assays for diagnosis infectious diseases. Rev. Sci. Tech. Off. Int. Epiz. 1998, 17, 469–486. [Google Scholar] [CrossRef]
  50. Jacobson, R.H. Principles of validation of diagnostic assays for infectious diseases. In OIE Manual of Diagnostic Tests and Vaccines for Terrestrial Animals, 5th ed.; Cambridge University Press: Cambridge, UK, 2004; Volume 1, pp. 21–29. [Google Scholar]
  51. Vizard, A.L.; Anderson, G.A.; Gasser, R.B. Determination of the optimum cut-off value of a diagnostic test. Prev. Vet. Med. 1990, 10, 137–143. [Google Scholar] [CrossRef]
  52. Ksiazek, T.G.; Rollin, P.E.; Williams, A.J.; Bressler, D.S.; Martin, M.L.; Swanepoel, R.; Burt, F.J.; Leman, P.A.; Khan, A.S.; Rowe, A.K.; et al. Clinical virology of Ebola hemorrhagic fever (EHF): Virus, virus antigen, and IgG and IgM antibody findings among EHF patients in Kiwit, Democratic Republic of the Congo, 1995. J. Infect. Dis. 1999, 179, S177–S187. [Google Scholar] [CrossRef] [PubMed]
  53. Wright, P.F.; Nilsson, E.; van Rooij, E.M.A.; Lelenta, M.; Jeggo, M.H. Standarization and validation of enzyme-linked immunosorbent assay techniques for the detection of antibody in infectious disease diagnosis. Rev. Sci. Tech. Off. Int. Epiz. 1993, 12, 435–450. [Google Scholar] [CrossRef]
  54. Krähling, V.; Becker, D.; Rohde, C.; Eickmann, M.; Eroglu, Y.; Herwig, A.; Kerber, R.; Kowalski, K.; Vergara-Alert, J.; Becker, S.; et al. Development of an antibody capture ELISA using inactivated Ebola Zaire Makona virus. Med. Microbiol. Immunol. 2016, 205, 173–183. [Google Scholar] [CrossRef]
  55. Sobarzo, A.; Perelman, E.; Groseth, A.; Dolnik, O.; Becker, S.; Lutwama, J.J.; Dye, J.M.; Yavelsky, V.; Lobel, L.; Marks, R.S. Profiling the native specific human humoral immune response to Sudan ebolavirus strain Gulu by chemiluminescence enzyme-linked immunosorbent assay. Clin. Vaccine Immunol. 2012, 19, 1844–1852. [Google Scholar] [CrossRef]
  56. Lopez, L.; Venteo, A.; Garcia, M.; Camuñas, A.; Ranz, A.; Garcia, J.; Sarraseca, J.; Anaya, C.; Rueda, P. Antigen-capture blocking enzyme-linked immunosorbent assay based on a baculovirus recombinant antigen to differentiate transmissable gastroenteritis virus from porcine respiratory coronavirus antibodies. J. Vet. Diagn. Investig. 2009, 21, 598–608. [Google Scholar] [CrossRef]
  57. Rowe, A.K.; Bertolli, J.; Khan, A.S.; Mukunu, R.; Muyembe-Tamfum, J.J.; Bressler, D.; Williams, A.J.; Peters, C.J.; Rodriguez, L.; Feldmann, H.; et al. Clinical, virologic and immunologic follow-up of convalescent Ebola hemorrhagic fever patientsand their household contacts, Kikwit, Democratic Republic of the Congo. J. Infect. Dis. 1999, 179, S28–S35. [Google Scholar] [CrossRef]
  58. Wauquier, N.; Becquart, P.; Gasquet, C.; Leroy, E.M. Immunoglobulin G in Ebola outbreak survivors, Gabon. Emerg. Infect. Dis. 2009, 15, 1136–1137. [Google Scholar] [CrossRef] [PubMed]
  59. Rudge, T.L., Jr.; Sankovich, K.A.; Niemuth, N.A.; Anderson, M.S.; Badorrek, C.S.; Skomrock, N.D.; Cirimotich, C.M.; Carol, L.; Sabourin, C.L. Development, qualification, and validation of the Filovirus Animal Nonclinical Group anti-Ebola virus glycoprotein immunoglobulin G enzyme-linked immunosorbent assay for human serum samples. PLoS ONE 2019, 14, e0215457. [Google Scholar] [CrossRef] [PubMed]
Viruses 11 00678 g001aViruses 11 00678 g001b
Figure 1. Distribution of percentage positivity (PP) values in South African (SA) and Sierra Leonean (SL) Ebola virus (EBOV) immunoglobulin G (IgG) negative human sera and selection of cut-off values for (A) the whole antigen (WAg), (B) nucleocapsid protein (NP) and (C) glycoprotein (GP) based ELISAs. Cut-off value for each assay was calculated as mean plus three standard deviations of ELISA PP (percentage positivity of internal positive control serum) values recorded by each test in 273 SA and 676 SL EVD negative sera, respectively.
Figure 1. Distribution of percentage positivity (PP) values in South African (SA) and Sierra Leonean (SL) Ebola virus (EBOV) immunoglobulin G (IgG) negative human sera and selection of cut-off values for (A) the whole antigen (WAg), (B) nucleocapsid protein (NP) and (C) glycoprotein (GP) based ELISAs. Cut-off value for each assay was calculated as mean plus three standard deviations of ELISA PP (percentage positivity of internal positive control serum) values recorded by each test in 273 SA and 676 SL EVD negative sera, respectively.
Viruses 11 00678 g001aViruses 11 00678 g001b
Viruses 11 00678 g002aViruses 11 00678 g002b
Figure 2. Optimization of cut-offs for Wag (A), nucleocapsid (B) and glycoprotein (C) ELISAs using the two-graph receiver operating characteristic analysis (TG-ROC). The insertion point of the sensitivity (Se, smooth line) and specificity (Sp, dashed line) graphs represents a cut-off PP value (13.53, 16.44 and 26.28, respectively) at which the highest and equivalent test parameters (Se = Sp) are achieved at 95% accuracy level. Using the misclassification cost term (MCT) option of the TG-ROC, at these cut-of values, the overall misclassification costs for WAg (A1), NP (B1), and GP ELISA (C1) become minimal (0.0003, 0.0069, 0.0001, respectively) under assumption of 50% disease prevalence and equal costs of false-positive and false-negative results. The two MCT curves represent values based on non-parametric (dashed line) or parametric (smooth line) estimates of Se and Sp derived from data sets analyzed. Optimization of cut-off values was based on the non-parametric program option due to departure from a normal distribution of data analyzed. Cut-off values are expressed as percentage positivity (PP) of an internal positive serum control.
Figure 2. Optimization of cut-offs for Wag (A), nucleocapsid (B) and glycoprotein (C) ELISAs using the two-graph receiver operating characteristic analysis (TG-ROC). The insertion point of the sensitivity (Se, smooth line) and specificity (Sp, dashed line) graphs represents a cut-off PP value (13.53, 16.44 and 26.28, respectively) at which the highest and equivalent test parameters (Se = Sp) are achieved at 95% accuracy level. Using the misclassification cost term (MCT) option of the TG-ROC, at these cut-of values, the overall misclassification costs for WAg (A1), NP (B1), and GP ELISA (C1) become minimal (0.0003, 0.0069, 0.0001, respectively) under assumption of 50% disease prevalence and equal costs of false-positive and false-negative results. The two MCT curves represent values based on non-parametric (dashed line) or parametric (smooth line) estimates of Se and Sp derived from data sets analyzed. Optimization of cut-off values was based on the non-parametric program option due to departure from a normal distribution of data analyzed. Cut-off values are expressed as percentage positivity (PP) of an internal positive serum control.
Viruses 11 00678 g002aViruses 11 00678 g002b
Viruses 11 00678 g003
Figure 3. Mean IgG responses in 423 Ebola disease patients measured by a whole antigen (WAg), nucleocapsid (NP), and glycoprotein (GP) indirect ELISAs at different time post disease onset.
Figure 3. Mean IgG responses in 423 Ebola disease patients measured by a whole antigen (WAg), nucleocapsid (NP), and glycoprotein (GP) indirect ELISAs at different time post disease onset.
Viruses 11 00678 g003
Viruses 11 00678 g004aViruses 11 00678 g004b
Figure 4. Dose-response kinetics of a positive EBOV IgG human serum before and after different inactivation procedures measured by a whole antigen (Wag), (A); nucleocapsid (NP), (B); and glycoprotein (GP), (C) indirect ELISA. 1 Percent positivity of internal positive control serum.
Figure 4. Dose-response kinetics of a positive EBOV IgG human serum before and after different inactivation procedures measured by a whole antigen (Wag), (A); nucleocapsid (NP), (B); and glycoprotein (GP), (C) indirect ELISA. 1 Percent positivity of internal positive control serum.
Viruses 11 00678 g004aViruses 11 00678 g004b
Table 1. Internal control quality data and repeatability estimates for a whole (WAg), nucleocapsid (NP), and glycoprotein (GP) indirect ELISAs for the detection of anti-IgG EBOV antibody in humans.
Table 1. Internal control quality data and repeatability estimates for a whole (WAg), nucleocapsid (NP), and glycoprotein (GP) indirect ELISAs for the detection of anti-IgG EBOV antibody in humans.
C++ 1OD 2 IQC 3Mean OD ± SD 4Mean CV 5 ± SD (%)
LCL 6UCL 7Intra-Plate VariationInter-Plate VariationIntra-Plate VariationInter-Plate Variation
WAg0.811.291.00 ± 0.091.00 ± 0.084.26 ± 2.674.41 ± 1.85
NP1.141.791.45 ± 0.181.50 ± 0.103.54 ± 1.993.11 ± 1.05
GP0.91.391.19 ± 0.091.2 ± 0.044.76 ± 2.164.8 ± 1.56
1 Positive internal control serum; 2 Optical density; 3 Internal quality control; 4 Standard deviation; 5 Coefficient of variations; 6 Lower control limit; 7 Upper control limit.
Table 2. Diagnostic specificity of whole antigen (WAg), nucleocapsid (NP), and glycoprotein (GP) indirect ELISAs for the detection of anti-IgG EBOV antibody in humans.
Table 2. Diagnostic specificity of whole antigen (WAg), nucleocapsid (NP), and glycoprotein (GP) indirect ELISAs for the detection of anti-IgG EBOV antibody in humans.
TestCut-off PP 1SA Negative Serum Panel (n = 273)SL Negative Serum Panel (n = 676)
FP 2/TN 3DSp 4 (95% CI 5)FP/TNDSp (95% CI)
WAg I-ELISA13.46 62/27199.3 (97.4–99.9)0/676100 (99.5–100)
WAg I-ELISA13.12 72/27199.3 (97.4–99.9)0/676100 (99.5–100)
WAg I-ELISA13.53 82/27199.3 (97.4–99.9)0/676100 (99.5–100)
NP I-ELISA16.13 61/27299.6 (98–100)0/676100 (99.5–100)
NP I-ELISA15.15 71/27299.6 (98–100)2/67499.7 (98.9–100)
NP I-ELISA16.44 81/27299.6 (98–100)0/676100 (99.5–100)
GP I-ELISA26.52 65/26898.2 (95.8–99.4)0/676100 (99.5–100)
GP I-ELISA25.12 77/26897.4 (94.8–99)2/67499.7 (98.9–100)
GP I-ELISA26.28 85/26898.2 (95.8–99.4)0/676100 (995–100)
1 Percentage of the positive internal control serum; 2 False positive; 3 True negative; 4 Diagnostic specificity; 5 Confidence intervals; 6 Cut-off value determined as mean + 3 standard deviations of PP values recorded in SA IgG EBOV negative serum panel; 7 Cut-off value determined as mean +3 standard deviations of PP values recorded in SL IgG EBOV negative serum panel; 8 Cut-off value at 95% accuracy level determined by the two-graph receiver operating characteristics analysis using data from SL IgG EBOV negative and positive serum panels.
Table 3. Results of a whole antigen (WAg), nucleocapsid (NP), and glycoprotein (GP) indirect ELISAs for the detection of IgG antibody in 423 Ebola disease cases bled at different times post disease onset.
Table 3. Results of a whole antigen (WAg), nucleocapsid (NP), and glycoprotein (GP) indirect ELISAs for the detection of IgG antibody in 423 Ebola disease cases bled at different times post disease onset.
Days Post Disease Onset (Number Individuals Tested)WAg No.pos. 1 (% pos.)NP No. pos.(% pos.)GP No. pos.(% pos.)Only WAg pos.Wag and NP pos.Wag and GP pos.Only NP pos.NP and GP pos.Only GP pos.
1
(n = 62)		3 2 (4.8)4 (6.5)1 (1.6)030010
3 3 (4.8)5 (8.1)2 (3.2)030111
3 4 (4.8)4 (6.5)1 (1.6)030010
2–4
(n = 119)		20 (16.8)27 (22.7)13 (11.0)160800
20 (16.8)28(23.5)13 (11.0)160900
20 (16.8)27 (22.7)13 (11.0)160800
5–7
(n = 93)		27 (29.0)34 (36.6)24 (25.8)170623
27 (29.0)34 (36.6)26 28.0)170534
27 (29.0)34 (36.6)24 (25.8)170623
8–14
(n = 61)		42 (68.9)49 (80.3)37 (60.7)180540
43 (70.5)50 (82.0)39 (64.0)190360
42 (68.9)49 (80.3)37 (60.7)180540
15–21
(n = 50)		50 (100)50 (100)49 (98.0)000000
50 (100)50 (100)49 (98.0)000000
50 (100)50(100)49 (98.0)000000
22–36
(n = 38)		38 (100)38 (100)37 (97.4)000000
38 (100)38(100)38 (100)000000
38 (100)38 (100)37 (97.4)000000
1 Positive result; 2 I-ELISA results using cut-off value determined as mean +3 standard deviations of ELISA readings recorded in South African IgG EBOV negative serum panel; 3 I-ELISA results using cut-off value determined as mean +3 standard deviations of ELISA readings recorded in Sierra Leonean IgG EBOV negative serum panel; 4 I-ELISA results using cut-off value determined by the two-graph receiver operating characteristics analysis of ELISA readings in Sierra Leonean IgG EBOV negative and positive serum panels.
Table 4. Diagnostic sensitivity of a whole antigen (WAg), nucleocapsid (NP), and glycoprotein (GP) I-ELISAs for the detection of anti-IgG EBOV antibody in humans.
Table 4. Diagnostic sensitivity of a whole antigen (WAg), nucleocapsid (NP), and glycoprotein (GP) I-ELISAs for the detection of anti-IgG EBOV antibody in humans.
TestCut-off
PP 1		SL EBOV RT-PCR Positive Cases
Days 8–36 Post Onset (n = 149)		SL EBOV RT-PCR Positive Cases
Days 15–36 Post Onset (n = 88)
FN 2/TP 3DSe 4 (95% CI 5)FN/TPDSe (95% CI)
WAg I-ELISA13.46 619/13087.2 (80.8–92.1)0/88100 (95.9–100)
WAg I-ELISA13.12 718/13187.9 (81.6–92.7)0/88100 (95.5–100)
WAg I-ELISA13.53 819/13087.2 (80.8–92.1)0/88100 (95.5–100)
NP I-ELISA16.13 612/13791.9 (86.4–95.8)0/88100 (95.5–100)
NP I-ELISA15.15 711/13892.6 (87.2–96.3)0/88100 (95.9–100)
NP I-ELISA16.44 812/13791.9 (86.4–95.8)0/88100 (95.5–100)
GP I-ELISA26.52 626/12382.6 (75.5–88.3)2/8697.7 (92–99.7)
GP I-ELISA25.12 723/12684.6 (77.7–90)1/8798.9 (93.8–100)
GP I-ELISA26.28 826/12382.6 (75.5–88.3)2/8697.7 (92–99.7)
1 Percentage of the positive internal control serum; 2 False negative; 3 True positive; 4 Diagnostic sensitivity; 5 Confidence intervals; 6 Cut-off value determined as mean +3 standard deviations of PP values recorded in SA IgG EBOV negative serum panel; 7 Cut-off value determined as mean + 3 standard deviations of PP values recorded in SL IgG EBOV negative serum panel; 8 Cut-off value at 95% accuracy level determined the two-graph receiver operating characteristics analysis using data from SL IgG EBOV negative and positive serum panels.
Table 5. Comparison of whole antigen (WAg), nucleocapsid (NP) and glycoprotein (GP) indirect ELISAs mean titers and analytical slopes of non-treated versus inactivated IgG EBOV positive serum.
Table 5. Comparison of whole antigen (WAg), nucleocapsid (NP) and glycoprotein (GP) indirect ELISAs mean titers and analytical slopes of non-treated versus inactivated IgG EBOV positive serum.
Assay
(Cut-off PP Value) 1		Mean log10 Serum Titre 2/Dose Response Curve R Square 3
Untreated60° 1 h0.5%Tween 20
15 min 60°		0.5%Triton X-100
15 min 60°
WAg I-ELISA
(13.53)		3.5/0.9453.5/0.9503.5/0.9423.5/0.952
NP I-ELISA
(16.44 PP)		3.8/0.9593.5/0.9613.5/0.9563.5/0.957
GP I-ELISA
(26.28)		3.2/0.9323.2/0.9423.2/0.9243.2/0.928
1 TG-ROC derived cut-off used; 2 Log10 highest serum dilution at which at least 50% of six replicates still tested positive; 3 Coefficient of determination.

Share and Cite

MDPI and ACS Style

Paweska, J.T.; Moolla, N.; Storm, N.; Msimang, V.; Conteh, O.; Weyer, J.; Vuren, P.J.v. Evaluation of Diagnostic Performance of Three Indirect Enzyme-Linked Immunosorbent Assays for the Detection of IgG Antibodies to Ebola Virus in Human Sera. Viruses 2019, 11, 678. https://doi.org/10.3390/v11080678

AMA Style

Paweska JT, Moolla N, Storm N, Msimang V, Conteh O, Weyer J, Vuren PJv. Evaluation of Diagnostic Performance of Three Indirect Enzyme-Linked Immunosorbent Assays for the Detection of IgG Antibodies to Ebola Virus in Human Sera. Viruses. 2019; 11(8):678. https://doi.org/10.3390/v11080678

Chicago/Turabian Style

Paweska, Janusz T., Naazneen Moolla, Nadia Storm, Veerle Msimang, Ousman Conteh, Jacqueline Weyer, and Petrus Jansen van Vuren. 2019. "Evaluation of Diagnostic Performance of Three Indirect Enzyme-Linked Immunosorbent Assays for the Detection of IgG Antibodies to Ebola Virus in Human Sera" Viruses 11, no. 8: 678. https://doi.org/10.3390/v11080678

APA Style

Paweska, J. T., Moolla, N., Storm, N., Msimang, V., Conteh, O., Weyer, J., & Vuren, P. J. v. (2019). Evaluation of Diagnostic Performance of Three Indirect Enzyme-Linked Immunosorbent Assays for the Detection of IgG Antibodies to Ebola Virus in Human Sera. Viruses, 11(8), 678. https://doi.org/10.3390/v11080678

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop