Large-Scale International Validation of an Indirect ELISA Based on Recombinant Nucleocapsid Protein of Rift Valley Fever Virus for the Detection of IgG Antibody in Domestic Ruminants

Diagnostic performance of an indirect enzyme-linked immunosorbent assay (I-ELISA) based on a recombinant nucleocapsid protein (rNP) of the Rift Valley fever virus (RVFV) was validated for the detection of the IgG antibody in sheep (n = 3367), goat (n = 2632), and cattle (n = 3819) sera. Validation data sets were dichotomized according to the results of a virus neutralization test in sera obtained from RVF-endemic (Burkina Faso, Democratic Republic of Congo, Mozambique, Senegal, Uganda, and Yemen) and RVF-free countries (France, Poland, and the USA). Cut-off values were defined using the two-graph receiver operating characteristic analysis. Estimates of the diagnostic specificity of the RVFV rNP I-ELISA in animals from RVF-endemic countries ranged from 98.6% (cattle) to 99.5% (sheep) while in those originating from RVF-free countries, they ranged from 97.7% (sheep) to 98.1% (goats). Estimates of the diagnostic sensitivity in ruminants from RVF-endemic countries ranged from 90.7% (cattle) to 100% (goats). The results of this large-scale international validation study demonstrate the high diagnostic accuracy of the RVFV rNP I-ELISA. Standard incubation and inactivation procedures evaluated did not have an adverse effect on the detectable levels of the anti-RVFV IgG in ruminant sera and thus, together with recombinant antigen-based I-ELISA, provide a simple, safe, and robust diagnostic platform that can be automated and carried out outside expensive bio-containment facilities. These advantages are particularly important for less-resourced countries where there is a need to accelerate and improve RVF surveillance and research on epidemiology as well as to advance disease control measures.


Introduction
The geographic expansion of Rift Valley fever virus (RVFV) in the last four decades associated with high health and socio-economic losses is of great concern for veterinary and public health worldwide. The wide distribution of potentially competent mosquito vectors in different geographic regions of the world and increased international trade and travel carry the risk of the introduction and spread of this zoonotic virus to RVF-free areas [1][2][3][4]. The unpredictable and sudden emergence of RVFV outside traditional endemic areas, unavailability of safe and efficacious antiviral treatment, and prophylactic immunization led the World Health Organization (WHO) to recognize RVF as a priority disease for the development of accurate diagnostics, effective therapeutics, and vaccines [5].
Clinical manifestations of Rift Valley fever (RVF) in livestock vary between species and depend largely on the age of the infected animal. Most severe symptoms are seen in small ruminants, where so-called "abortion storms" may result in very high fetal and neonatal losses [6,7]. Clinical signs in humans vary from mild flu-like conditions to meningoencephalitis, retinitis, and hemorrhagic fever syndrome [8,9]. RVFV is suspected to induce miscarriages in women [10]. RVFV belongs to a group of viral hemorrhagic fever (VHF) agents regarded as a potential bioweapon with high adverse impacts on public health and agriculture [11,12]. As for most VHFs, the non-specific presentation of RVF makes it difficult to diagnose clinically. Therefore, the differential diagnosis in both humans and animals concerns a broad array of conditions, especially when first cases are encountered during a yet unrecognized outbreak. RVF may be suspected when there is a sudden outbreak of febrile illness with headache and myalgia in humans, in association with the occurrence of abortions in domestic ruminants and deaths of young animals following heavy rains [3,6,9]. RVFV is transmitted among animals mostly by aedine and culicine mosquitoes. Current data suggest that over 50 mosquito species, many of which have global distribution, can potentially act as vectors of RVFV [13,14]. Humans usually become infected following contact with virus-contaminated tissues and body fluids from infected animals, but mosquito bites can also transmit the virus [15][16][17].
RVFV is a negative-stranded RNA virus, a member of the genus Phlebovirus, family Phenuiviridae. The genome of RVFV comprises three segments, encoding the RNAdependent RNA polymerase (L segment), the two surface proteins Gn and Gc as well as the nonstructural protein NSm (M segment), the nucleoprotein (NP), and a further nonstructural protein NSs (S-segment) [18]. The N protein is the most abundant protein in phlebovirus-infected cells and strongly immunogenic [19,20].
Various diagnostic methods are available for laboratory confirmation of infections with RVFV. Isolation of RVFV is achieved in hamsters, infant or adult mice, and various cell cultures [6,21]. Highly sensitive genetic amplification assays for the detection and quantification of RVFV in serum and other tissues of infected humans and livestock, as well as mosquitoes, have been reported. These assays include polymerase chain reaction (PCR) [22][23][24][25], real-time RT PCR (RTD-PCR) based on TaqMan probe technology [26,27], and the real-time reverse-transcription loop-mediated isothermal amplification assay [28]. A viral antigen can be detected in blood and other tissues by a variety of immunological methods, including agar gel immunodiffusion and immunostaining assays [6,14,29]. A sandwich ELISA was developed for the detection of the nucleocapsid protein of RVFV in various clinical specimens [30]. The lateral flow immunochromatographic test for the detection of RVFV NP in animal sera and fluids from aborted fetuses provides a valuable diagnostic tool for onsite rapid detection of the virus [31].
Although viremia in infected individuals reaches high titers, it is of short duration, thus limiting the use of viral antigen and molecular detection systems [13]. The collection of diagnostic specimens after viral clearance and inappropriate transportation methods and storage conditions may negatively affect molecular assays, making serology testing an important diagnostic capacity in the veterinary and public health response to outbreaks occurring in remote locations where limited resources are available. Most adult animals and infected patients undergo subclinical or mild infections; therefore, antigen and nucleic acid detection assays should be run in parallel with antibody-detecting techniques. Typespecific antibodies to RVFV are easily demonstrable shortly after exposure to the virus. Serodiagnosis of recent infection can be confirmed by demonstrating seroconversion or a fourfold or greater rise in titer of the antibody in paired serum samples or by the detection of IgM antibody [9,13].
The classical methods for the detection of antibodies to RVFV include hemagglutination inhibition, complement fixation, indirect immunofluorescence, and virus neutralization tests [6,14,32]. Although regarded as a gold standard, the virus neutralization tests are laborious, expensive, and require 5-7 days for completion. These assays can be performed only when a standardized stock of live virus and tissue cultures are available, which poses a health risk to laboratory personnel [33,34], thus restricting their use outside RVF-endemic areas and/or to high bio-containment facilities. Consequently, virus neutralization assays are rarely used, and then only in highly specialized reference laboratories [14]. To address this issue, a virus neutralization test based on an avirulent RVFV expressing an enhanced green fluorescent protein was developed and reported to be more sensitive than the classical neutralization test [35]; however, this test takes multiple days to perform.
Various enzyme-linked immunosorbent assay (ELISA) formats have been developed and validated in recent years for the specific detection of anti-RVFV antibodies in humans and animals, based on β-propiolactone inactivated or gamma-irradiated antigens derived from infected tissue cultures or mouse liver [36][37][38][39][40][41]. An optical fiber immunosensor based on a sandwich ELISA using a gamma-irradiated RVFV whole antigen was developed for the detection of the anti-RVFV IgG antibody in human sera [42]. While ELISAs based on an inactivated whole antigen of RVFV have high diagnostic accuracy compared to virus neutralization assays [36][37][38][39][40][41], the production of antigens for these assays also requires high bio-containment facilities. Recombinant antigen technology allows the production of high-quality viral antigens under biosafety level two conditions. Indirect ELISAs based on the recombinant NP or GP proteins for the detection of anti-RVFV antibodies have been developed [43][44][45][46]. An ELISA platform based on recombinant NP and NSs proteins can distinguish infected from vaccinated animals. Given very strict regulations for the international trade of animals from RVF-endemic countries, this diagnostic tool has the potential to assist in the safe movement and trade of domestic ruminates [47]. A multiplex fluorescence microsphere immunoassay (FMIA) was developed to detect IgM and IgG antibodies in ruminant sera to RVFV structural and non-structural proteins. Preliminary results demonstrate the potential of the FMIA diagnostic platform for the development of diagnostic tests that can be used to differentiate vaccinated from infected animals and for simultaneous differential diagnosis of several abortive and zoonotic pathogens [48,49].
The increasing importance of RVF as a zoonotic threat and the need for a better understanding of RVF epidemiology requires the application of well-evaluated tools for antibody detection across different geographic areas and domestic and wildlife ruminants implicated in RVF epidemiology. Reasons for the test validation include the need for a statistically sound evaluation of assay diagnostic performance parameters that are essential for reporting diagnostic results, comparing results between different diagnostic laboratories, determining seroprevalence rates, designing infection risk population studies, and assessing the occurrence of asymptomatic infections. The process of an ELISA standardization and validation is, however, complex, time-consuming, expensive, and vulnerable to many limitations, including the availability of recommended standards and representatives of the reference sera [61][62][63][64].
In response to the increasing demand for safe serological diagnosis of RVF, we validated the diagnostic performance of RVFV rNp-based I-ELISA for the detection of IgG antibodies using large serum panels collected from sheep, goats, and cattle in RVF-endemic and RVF-free countries.

Origin of Serum Specimens
A total of 9818 individual banked sera from sheep (n = 3367), goats (n = 2632), and cattle (n = 3819), collected in RVF-endemic (n = 6810) and RVF-free countries (n = 3008) in 2000-2009, were used (Table 1). These sera represent submissions to national veterinary laboratories for routine diagnostic testing. They were shipped to South Africa on ice packs under veterinary import permits for the importation of diagnostic specimens, issued by the Directorate of Animal Health, Department of Agriculture, South Africa (Permit Nos.: 13/1/30/4-131 and 13/1/1/30/2/1/20-341). Upon shipment, sera were stored at −70 • C at the National Institute for Communicable Diseases until tested. The RVF vaccination or infectious status of sampled animals was unknown.

VNT
A virus neutralization test (VNT) was used to dichotomize sera according to RVFV infection status. Each serum was tested in duplicate in VNT as described previously using the AR 20368 strain of RVFV isolated in 1981 from Culex zombaensis in South Africa [38]. Titers were expressed as the reciprocal of the serum dilution that inhibited ≥75% of the viral cytopathic effect. A serum sample was considered positive for anti-RVFV neutralizing antibodies when it had a titer of ≥1:10.

rNP I-ELISA
Previously produced freeze-dried, gamma-irradiated sheep serum controls were used [38,58]. The production of rNP antigen for I-ELISA in E. coli expression system and the assay procedure were carried out as described previously [44] with minor modifications. The NP sequence is based on the Zim688/78 RVFV strain isolated from a bovine in 1978 in Zimbabwe [44]. The coding sequence is available in Genbank, accession number DQ924959. Briefly, all control and test sera and reagents were added to microtiter 96-wells plates (MaxiSorb Immunoplates, Nunc, Roskild, Denmark) at a volume of 100 µL/well unless otherwise stated. The passive absorption of stock RVFV rNP antigen was performed at 4 • C overnight in carbonate/bicarbonate buffer (pH 9.6), and all subsequent incubations (except for substrate addition) were performed at 37 • C in a humidified chamber for 1 h. Following coating, plates were washed three times with 300 µL/well of phosphate-buffered saline (PBS) pH 7.2 and 0.1% Tween; the same washing procedure followed each subsequent stage of I-ELISA. Plates were blocked with 200 µL/well 10% fat-free milk powder in PBS. After incubation, plates were washed, and control and test sera diluted 1:400 in PBS containing 2% milk powder (diluent buffer) were added to the plates. Each test serum was assayed in duplicate and each internal control was tested in quadruplicate. After incubation, plates were washed and the HRPO conjugated Protein G (Zymed Laboratories, Inc., South San Francisco, CA, USA), diluted 1:5000 in diluent buffer, was added to the wells. Afterward, incubation plates were washed and 2,2 -azinodiethylbenzothiazoline sulfonic acid peroxidase (Seracare Lifesciences, Milford, MA, USA) substrate was added to wells. Plates were incubated in the dark at room temperature (±22 • C) for 30 min. Reactions were stopped by the addition of 1% sodium dodecyl sulfate, and the optical density readings (OD) were measured at 405 nm. OD readings were converted into a percentage positivity (PP) of high-positive control serum (C++) using the following equation: [PP = (mean test serum OD/mean OD C++) × 100] [36].

Selection of Cut-Off Values
Optimization of rNP I-ELISA cut-off PP values at 95% diagnostic accuracy level were performed using the misclassification costs term (MCT) option [65] of the twograph receiver operating characteristics (TG-ROC) analysis. TG-ROC is a Microsoft EXCEL spreadsheet template designed for selecting cut-off values in quantitative diagnostic tests at a preselected accuracy level (e.g., 90% or 95% sensitivity and specificity) [66,67]. The optimization of cut-off values was based on the following equation: where p (prevalence) = 0.5, r (costs of false-positive and false-negative results) = 1, Sp = specificity, and Se = sensitivity [65]. In addition, the cut-off values were determined as the mean plus two standard deviations (mean + 2SD) as well as by the mean plus three standard deviations (mean + 3SD) of the results observed with the RVF-free subpopulations (VNT-negative animals).

Determination of Diagnostic Accuracy and Other Statistical Analysis
Estimates of diagnostic sensitivity (DSn) and diagnostic specificity (DSp) and their 95% confidence intervals (CI), Youden's index (J), efficiency (Ef), positive predictive value (PPV), and negative predictive value (NPV) were calculated as previously described [39,64]. Means and standard deviations and ranges of I-ELISA PP values of the rNP I-ELISA were calculated by endemicity and by country. Based on the Shapiro-Wilk test that determined if the samples came from a normal distribution, a non-parametric Mann-Whitney U-test was performed to assess whether there was a significant difference between the median PP values of RVF-endemic and RVF-free countries by species of sheep, goats, or cattle. Using Duncan's test [68] for descriptive statistics, we determined which pairs of means resulting from the country comparison were significantly different from each other by livestock species tested. All calculations and tests were performed using Stata, and a P-value lower than 0.05 was considered statistically significant (Stata Corporation, College Station, TX, USA).

Serum Inactivation
To assess the effect of heat and chemical inactivation on the levels of the detectable anti-RVFV IgG by rNP I-ELISA in ruminant sera, laboratory protocols previously shown to completely inactivate RVFV [30] and other highly hazardous RNA viruses were used [69,70]. The high positive sheep, goat, and cattle sera were first diluted 1:10 in species-corresponding untreated negative serum containing either 0.5% Triton X-100 or 0.5% Tween 20 (Sigma-Aldrich, Taufkichen, Germany) and heated at 60 • C for 15 min. Then, two-fold log 10 dilutions (from log 10 10 2 − log 10 10 5.1 ) of untreated and treated sera were tested. Each serum dilution was tested in duplicate on 3 separate runs. A serum titer was considered the highest sample dilution at which its PP value was ≥ the rNP I-ELISA cut-off.

rNP I-ELISA Robustness
To assess the robustness of the rNP I-ELISA, the assay was performed using three different incubation conditions: (1) the ELISA plate was coated with rNP overnight at 4 • C, and all subsequent incubations (except for substrate addition) were performed at 37 • C for 1 h (as per the standardized protocol described in Section 2.4); (2) the ELISA plate was coated with rNP at 37 • C for 1 h and all subsequent incubations (except for substrate addition) were carried out at 37 • C for 1 h; (3) all incubations were performed at room temperature for 1 h, including coating with rNP. For each of the three different assay incubation conditions, four individual sera of each species, each representing different levels of anti-RVF IgG antibody, were tested in quadruplicate.

IgG-Sandwich RVFV ELISA
To compare results between ELISAs based on a recombinant and based on a whole RVFV antigen, a subset of 27 sheep sera from the USA that tested negative by VNT but positive by IgG rNP I-ELISA when using the TG-ROC derived cut-off was assayed using an IgG-sandwich ELISA [38].

VNT
Of the 9818 sera tested, 8818 were negative and 1000 were positive by VNT. All VNTpositive sera were from RVF-endemic countries. Irrespective of the geographic origin, all VNT-negative sera were regarded in this study as reference panels from RVFV non-infected animals, and VNT-positive sera were considered reference panels from animals infected with RVFV (Table 1).

Selection and Optimization of Cut-Off Values
Selection and optimization of cut-offs for the RVF IgG rNP I-ELISA in sheep, goats, and cattle using the TG-ROC and MCT analysis are shown in Figure 1. The optimization of cut-offs was based on the non-parametric program option [61,63]

Distribution of rNP I-ELISA PP Values in Ruminant Sera from RVF-Endemic Countries and the Selection of Cut-Offs
The distribution of rNP I-ELISA PP values in VNT-negative and VNT-positive domestic ruminant sera from RVF-endemic countries and graphic illustration of the effect of different cut-offs on the test results are shown in Figure 3. Irrespective of the ruminant species tested and cut-off used, there was an overlap of PP I-ELISA values between VNT-positive and VNT-negative sera. The lowest number of false-positive results in VNTnegative sera was at TG-ROC selected cut-off and the highest at cut-off determined as mean + 2SD.

D-Se and D-Sp
Estimates of D-Se at 95% and other measures of combined diagnostic accuracy of RVF rNP I-ELISA in ruminants from RVF-endemic countries are given in Table 2. The assay had the lowest estimates of D-Se in cattle, ranging from 90.7% to 95.8%, and was the lowest when TG-ROC and mean + 3SD derived cut-offs were applied. At all cut-offs, the D-Se in goats was 100%. The D-Se in sheep ranged from 97.5% to 99.6% and was the highest when mean + 2SD cut-off was applied.  9 Negative predictive value; 10 Cut-off value determined as mean + 2 standard deviations of PP values in VNT-negative animals; 11 Cut-off value determined as mean + 3 standard deviations of PP values in VNT-negative animals; 12 Cut-off value determined by the two-graph receiver operating characteristic analysis in ruminant serum panels from RVF-endemic countries.
The Ef estimates ranged from 0.947 to 0.967 and irrespective of the cut-off used were the lowest in cattle. Y estimates ranged from 0.893 to 0.988 and were the lowest in cattle at all cut-offs used. PPV estimates ranged from 83.8% to 97.12%, and all animals were the lowest when mean + 2SD cut-offs were applied. NPV estimates ranged from 97.2% to 100%, with the lowest in cattle and the highest in goats, irrespective of the cut-off used.
Estimates of D-Sp in subpopulations of ruminants from different RVF-endemic and RVF non-endemic countries are given in Table 3. In sheep from RVF-endemic countries, the D-Sp ranged from 92.7% to 96.6% at cut-off mean+2SD; from 97.6% to 98.4% at cut-off mean + 3SD; and from 98.6% to 100% at the TG-ROC-derived cut-off. Depending on the cut-off used, in sheep from RVF-non-endemic countries, the D-Se ranged from 96% to 98.9%. For the total of sheep serum panels from RVF-endemic countries, the D-Se ranged from 95.7% to 99.5%; for RVF non-endemic countries, it ranged from 96.5% to 97.7%; and for all sheep serum panels, it ranged from 96.7% to 98.6%, with the highest estimates of D-Sp recorded for all total panels when the TG-ROC derived cut-off was used (Table 3).
In goats from RVF-endemic countries, the D-Sp ranged from 94.7% to 95.7% at cut-off mean + 2SD, from 97.4% to 97.8% at cut-off mean + 3SD, and from 98.3% to 100% at TG-ROC derived cut-off. In a single panel of goat sera from RVF non-endemic countries, the D-Se ranged from 98.4% to 99.1%. For the total of goat serum panels from RVF-endemic countries, the D-Se ranged from 95.2% to 98.8%, and for all goat serum panels, it ranged from 96.5% to 98.9%, with the highest estimates of D-Sp recorded for all total panels when the TG-ROC derived cut-off was used (Table 3).
In cattle from RVF-endemic countries, the D-Sp ranged from 95.0% to 96.8% at cut-off mean + 2SD, from 96.8% to 98.86% at cut-off mean + 3SD, and from 93.6% to 100% at TG-ROC derived cut-off. Depending on the cut-off used, in sheep from RVF non-endemic countries, the D-Se ranged from 94.2% to 100%. For the total of sheep serum panels from RVF-endemic countries, the D-Se ranged from 96.3% to 98.7%; from RVF non-endemic countries, it ranged from 96.7% to 98.6%; and for all sheep serum panels, it ranged from 96.6% to 98.7%, with the highest estimates of D-Sp recorded for all total panels when the mean + 3SD cut-off was applied (Table 3). The Mann-Whitney U-test for statistical analysis of the distribution of I-ELISA PP values showed that there were highly significant differences (p < 0.001) between PP values of subpopulations from the RVF-endemic and RVF-free groups per animal species of sheep, goats, or cattle ( Table 4). The descriptive intercountry RVF-endemic pairwise comparison showed that all countries were significantly different except for Mozambique and Burkina Faso, which had similar PP means for each species of sheep, goats, or cattle. In addition, Yemen and Burkina Faso or Mozambique had similar PP means for sheep or cattle and Yemen and Uganda for goats.

The Effect of Inactivation
The different physicochemical inactivation methods used and evaluated in this study did not have an adverse effect on the kinetics and the detectable levels of the anti-RVFV IgG in positive ruminant sera. The titers (Table 5) as well as the dynamics of the dose-response curves ( Figure 5) were similar before and after inactivation.

The Effect of Temperature Incubation
The PP values for high positive sera (S1-2, G1-2, and C1-2), mid-positive (S3, G3, and C3), and negative sera (S4, G4, and C4) were similar but slightly lower when all ELISA incubations were done at 37 • C for 1 h. Nonetheless, irrespective of the incubation method used, the assay was highly repeatable as evidence by low coefficient of variance (CV) values and with the use of different cut-offs determined in this study; all tested sera are correctly classified as positive and negative (Table 6).  4 Each of four sheep (S1-4), goat (G1-4), and cattle (C1-4) sera were tested in quadruplicate; 5 mean percentage positivity (PP) value of four replicates; 6 standard deviation; 7 coefficient of variation.

IgG-Sandwich RVFV ELISA
Of 27 North American sheep sera that yielded negative results in VNT but positive results in IgG rNP I-ELISA when using TG-ROC derived cut-off (Table 3), 25 (92.6%) tested negative and two (7.4%) positive by IgG-sandwich RVFV ELISA.

Discussion
The continuous threat of re-emergence of large RVF outbreaks with dramatic veterinary and public health and socio-economic impacts in regions of endemicity and the risk of RVFV spread to RVF-free areas requires the development and application of safe, robust, high-throughput, and accurate diagnostic tests [71,72].
Diagnostic laboratories are tasked with providing analytical results that are based on internationally recognized methods and standards [73,74]. Assay validation is a part of quality assurance aiming at safeguarding qualification and competency and consequently indispensable in achieving accreditation status by an analytical laboratory [75]. A validated serological assay consistently provides test results that identify animals as seronegative or seropositive and by inference predicts the infection status of animals with a predetermined level of statistical certainty [62]. The epidemiology of RVF in endemic areas is still poorly understood and there is a need to evaluate low-cost surveillance tools, particularly for lowand middle-income settings (LMICs) [60].
Traditional antigen production methods pose health risks, thus restricting their use to high biosafety level facilities. Furthermore, due to the poor binding of these antigens to ELISA plates, it is necessary to use more complex and time-consuming ELISA formats, including sandwich, capture, or inhibition ELISA techniques [38,40]. In addition, the binding of antibodies to cellular contaminants present in RVFV whole-antigen preparations may lead to cross-reactivity, resulting in reduced specificity. To overcome these limitations, high-quality recombinant antigens can be safely produced outside high-bio-containment facilities.
An I-ELISA represents the most simple, cost-effective, and easy-to-automate immunoassay [76]. The RVFV NP is a supreme antigen for I-ELISA due to its high immunogenicity [19,20] and high nucleic acid conservation [14,77]. Antigenic cross-reactivity studies using sera from experimentally infected sheep demonstrated that antibodies induced by African phleboviruses other than RVFV should not confuse serodiagnosis of RVF [78]. RVFV rNP-based I-ELISA has been reported to not cross-react with sera from mice experimentally infected with different viruses of genus Phlebovirus, Nairovirus and Orthobunyavirus [57]. Other advantages of RVFV rNP-based I-ELISA, including cost-effectiveness of rNP antigen production, have been previously discussed [43,44].
Ideally, the diagnostic performance of a serological assay should be determined by testing sera from subjects of known infection status. The diagnostic threshold or cutoff represents an assay value used to dichotomize negative and positive results and, by inference, to define the infection status of an individual. The relevance of data used for the determination of cut-off consequently affects estimates of test performance. Gold standards for the selection of truly infected and uninfected subjects include isolation of the agent or pathognomonic histopathological criteria. In practice, a true gold standard is difficult to accomplish; thus, relative standards of comparison are used [62]. In this study, we used the VNT test to define the RVFV infection status of animals tested. Although virus neutralization assays are generally considered reference tests and are still used in reference diagnostic laboratories, they are labor-intensive, time-consuming, and require expensive bio-containment facilities [14]; thus, they are not easily affordable and impractical for surveillance activities, particularly in LMICs [60].
Various statistical analyses used in our study for the selection of the cut-off values provided similar results. A cut-off value determined as two or three SD above the mean in uninfected individuals is frequently used for the interpretation of serodiagnostic tests. However, this approach assumes a normal distribution of the test values in the population targeted by an assay and provides only an estimate of D-Sp [62]. Deviations from normality are often recorded in serological data and should be addressed in the selection of diagnostic threshold values [79]. Therefore, we also used the TG-ROC analysis for the selection and optimization of cut-off values to account for parametric versus nonparametric distribution of test values. Due to the inherent differences amongst assay systems, binding-antibody levels should be expressed in relative rather than absolute terms. One of the advantages of conversion OD readings into PP values of the positive serum standard as a measure of antibody activity in the I-ELISA is that this method does not assume a uniform background activity, and therefore it is more appropriate for inter-laboratory standardization [61].
We analyzed and compared the diagnostic performance of RVFV rNP-based I-ELISA in geographically separate populations and different domestic ruminant species in both RVF-endemic and RVF-free countries. The results demonstrate that the test performs consistently across different animal groups investigated with high estimates of D-Sp and D-Se and other measures of diagnostic accuracy. The observed variations across the different subpopulations tested may be due to the relatively small numbers of animals representing each specific subpopulation. To account for the distribution of covariate factors (age, sex, genetic, nutritional, geographical, and stage of infection) that may influence the estimates of an ELISA diagnostic accuracy, the targeted populations of animals should preferably be sampled using simple random, systematic, or stratified sampling methods [64]. These ideal conditions are difficult to achieve in practice and could not be applied during this study. Sera used in this study represent referral diagnostic submissions, and except for their animal and geographic origins and categorization by the VNT results, no other covariate factors that might influence the estimates of performance characteristics of the RVFV rNP-based I-ELISA were analyzed. While the influence of referral submissions on the performance of a diagnostic test under validation should be considered, this seems to not apply to collections of sera from laboratories that are involved in large-scale routine testing [63]. Time-dependent changes in the sensitivity of an assay may be of significance for epidemic situations, where the stage of disease may affect the outcome of assay results and interpretation of data. However, in practice, the impacts and interrelationship of multiple factors are mostly unknown, and assay diagnostic accuracy estimates are based on average values calculated in non-homogeneous populations. Biological variables possibly contribute more significantly to false-positive than to false-negative results [62].
Therefore, to account for the probable increased variance that would affect the estimates of the diagnostic specificity, relatively large numbers of animals from RVF-free countries were analyzed in the present study. From the point of using the VNT in classifying animals as infected or non-infected, it should be noted that infection with RVFV induces rapid appearance and long-lasting IgG antibodies following RVFV infection [9,14], making their detection a useful tool for diagnosis of outbreaks, epidemiological investigations, and disease risk studies. There is also no evidence of serological subgroups or major antigenic variations between RVFV isolates of disparate chronologic or geographic origins [6]. In this context, the use of the serological gold standard in dichotomizing animals according to their infection status and the relatively large total numbers of animals from RVF-endemic and RVF-free areas seems to ensure statistically reliable estimates of rNP-based I-ELISA diagnostic performance.
In this study, the TG-ROC MCT-optimized cut-offs were selected under the assumption that the cost of false-positive and false-negative results are equal and the prevalence of infection was 50%. The prevalence assumed in our study may not be the same in other populations targeted by the assay, and this should be noted when applying the estimates of diagnostic accuracy reported for the RVFV rNP-based I-ELISA in the present work.
The diagnostic accuracy estimates determined in our study are similar to those previously published for ELISAs based on a whole RVFV antigen [36][37][38][39][40][41]55] as well as those based on recombinant NP ELISAs [51][52][53][54][55][56][57][58][59][60], including the commercially available NP-based competition ELISAs [52,60]. Although recorded at a relatively low rate, false-positive results were documented in this study across different subpopulations and are of particular concern in ruminants originating from RVF-non-endemic countries. A similar problem has been reported in North American sheep [46]. The reasons for the anti-RVFV NP cross-reacting IgG antibodies remain unknown, but it has been postulated that potential cross-reactivity might be caused by unidentified closely related agent(s) circulating among ruminants in RVF-free areas or potential cross-reaction with antibodies raised against commensal or pathogenic E. coli in ruminants [46]. Both assumptions seem to be supported by the results of our study. While most of VNT-negative sheep sera from North America that tested positive by rNP-based I-ELISA were negative by IgG-sandwich RVFV ELISA, 7.4% tested positive by the latter assay, which is based on a whole antigen of RVFV [38]. The lower D-Se of rNP-based I-ELISA in cattle recorded in our study was also reported for a commercial competitive ELISA based on recombinant NP in Cameroonian cattle with D-Se ranging from 84.4% to 98.1% between different subpopulations tested [60]. The reason for lower D-Se remains unknown but is likely due to the higher cross-reactiveness of cattle sera.
Unlike E. coli, baculovirus seems to be not infectious for ruminant domestic livestock, which makes it an appropriate virus for the production of recombinant animal diagnostic antigens. Promising preliminary performance of the RVFV I-ELISA based on the use of the baculovirus-expressed recombinant NP antigen was recently reported in sera from RVFV experimentally infected sheep and calves and in sera from indigenous sheep and goats naturally infected in the Gambia [46].
Inactivation protocols evaluated in our study indicate that they do not affect detectable levels of anti-RVFV-IgG. Likewise, different incubation procedures evaluated in this study did not have an adverse effect on the detectable levels of the anti-RVFV-IgG in ruminant sera, repeatability, or interpretation of the test results.
Achieving the required level of assay validation is impossible without international collaboration and willingness to share clinical materials between countries both from disease-endemic and non-endemic regions. This work constitutes the largest validation study undertaken for rNP-based I-ELISA to date and significantly contributes to researchcollaborating activities aiming at improving surveillance tools for zoonotic pathogens of global concern and their application in less-resourced laboratories.
In summary, the performance characteristics of RVFV rNP-based I-ELISA determined in this study further demonstrate that the assay is a valuable surveillance tool for the detection of IgG anti-RVFV in domestic ruminants. The relatively low-cost and easyto-perform I-ELISA format makes it a suitable diagnostic test for LMICs. The standard incubation and inactivation procedures evaluated did not have an adverse effect on the detectable levels of the anti-RVFV IgG in ruminant sera and thus together with recombinant antigen-based I-ELISA provide a simple, safe, and robust diagnostic platform.