Next Article in Journal
Integrated Transcriptome and Proteome Analysis Provides Insight into the Ribosome Inactivating Proteins in Plukenetia volubilis Seeds
Next Article in Special Issue
SARS-CoV-2: Advances in Serological Methods and the Understanding of Antibody Escape
Previous Article in Journal
Interplay between the Chd4/NuRD Complex and the Transcription Factor Znf219 Controls Cardiac Cell Identity
Previous Article in Special Issue
SARS-CoV-2 and Immunity: Natural Infection Compared with Vaccination
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
IJMS
Volume 23
Issue 17
10.3390/ijms23179566
ijms-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Correlation between In Vitro Neutralization Assay and Serological Tests for Protective Antibodies Detection

by
Maria Addolorata Bonifacio
1,†,
Riccardo Laterza
1,†,
Angela Vinella
1,
Annalisa Schirinzi
2,
Mariangela Defilippis
3,
Francesca Di Serio
2,
Angelo Ostuni
3,
Antonio Fasanella
4 and
Maria Addolorata Mariggiò
1,*
1
Department of Biomedical Sciences and Human Oncology, University of Bari Aldo Moro Medical School, 70124 Bari, Italy
2
Clinical Pathology Unit, University of Bari Aldo Moro, 70124 Bari, Italy
3
Immunohematology and Transfusion Medicine Service, Azienda Ospedaliero-Universitaria Consorziale Policlinico di Bari, University of Bari “Aldo Moro”, 70124 Bari, Italy
4
Istituto Istituto Zooprofilattico Sperimentale della Puglia e della Basilicata, 71121 Foggia, Italy
*
Author to whom correspondence should be addressed.
†
These authors equally contributed to the work.
Int. J. Mol. Sci. 2022, 23(17), 9566; https://doi.org/10.3390/ijms23179566
Submission received: 21 July 2022 / Revised: 20 August 2022 / Accepted: 21 August 2022 / Published: 24 August 2022
(This article belongs to the Special Issue SARS-CoV-2 Variants, Antibodies and Vaccines)
Browse Figures
Versions Notes

Abstract

:
Serological assays are useful in investigating the development of humoral immunity against SARS-CoV-2 in the context of epidemiological studies focusing on the spread of protective immunity. The plaque reduction neutralization test (PRNT) is the gold standard method to assess the titer of protective antibodies in serum samples. However, to provide a result, the PRNT requires several days, skilled operators, and biosafety level 3 laboratories. Therefore, alternative methods are being assessed to establish a relationship between their outcomes and PRNT results. In this work, four different immunoassays (Roche Elecsys® Anti SARS-CoV-2 S, Snibe MAGLUMI® SARS-CoV-2 S-RBD IgG, Snibe MAGLUMI® 2019-nCoV IgG, and EUROIMMUN® SARS-CoV-2 NeutraLISA assays, respectively) have been performed on individuals healed after SARS-CoV-2 infection. The correlation between each assay and the reference method has been explored through linear regression modeling, as well as through the calculation of Pearson’s and Spearman’s coefficients. Furthermore, the ability of serological tests to discriminate samples with high titers of neutralizing antibodies (>160) has been assessed by ROC curve analyses, Cohen’s Kappa coefficient, and positive predictive agreement. The EUROIMMUN® NeutraLISA assay displayed the best correlation with PRNT results (Pearson and Spearman coefficients equal to 0.660 and 0.784, respectively), as well as the ROC curve with the highest accuracy, sensitivity, and specificity (0.857, 0.889, and 0.829, respectively).

1. Introduction

Since the declaration of the COVID-19 pandemic status by the World Health Organization (WHO), different serological tests to detect antibodies against SARS-CoV-2 have been marketed worldwide [1,2]. Serological tests commercialized so far are based on different reaction chemistries (i.e., chemiluminescent/electro-chemiluminescent, immunochromatographic, and immunoenzymatic methods) and claim the qualitative or quantitative determination of antibodies against viral antigens [3,4]. The number of performed serological tests increased during the first COVID-19 wave to support the study of virus spread. Later on, serological tests were carried out in an attempt to predict the duration of vaccine-induced immunity to address future vaccination strategies and, ultimately, to estimate the protective humoral immunity within a population [5,6]. In February, the European Centre for Disease Prevention and Control (ECDC) recommended the use of serological tests for population surveys in the context of epidemiological studies [7]. Conversely, the prediction of protective immunity has been discouraged on a single patient scale by the same ECDC. Several studies highlight that the immune response, with humoral immunity tightly bound to cell-mediated response, cannot be reduced only to antibody synthesis [8,9]. Indeed, the immune response is a complex and dynamic process, characterized by inter- and intra-individual variability, of which antibodies represent just one of many faces [10].
Nevertheless, recent seroepidemiological studies have reported the opportunity to analyze serological data to gain insights into immune protection from SARS-CoV-2 infection [11,12]. The titers of antibodies with neutralizing activity (NAb) could be determined by the plaque reduction neutralization test (PRNT50), which is considered the reference method [13,14]. However, the PRNT50 has several disadvantages, i.e., it requires biosafety level 3 laboratories to handle the virus alive, as well as properly skilled operators. Furthermore, it is expensive and time-consuming [15]. Therefore, in an effort to understand the kinetics and dynamics of protective humoral immunity development, surrogate viral neutralization tests (sVNT) have been marketed based on the competitive incubation of patient’s sera and RBD spike proteins with human angiotensin-converting enzyme 2 (ACE2) receptor [16,17]. As a further step, the relation between sVNT outcomes, in vitro PRNT, and immunological tests is being intensively explored [18,19,20].
The aim of this work consists of the study of humoral immunity in a homogenous set of 83 patients severely infected by SARS-CoV-2 at least 14 days after a negative molecular test. Serum samples were tested by the reference method, i.e., PRNT50, as well as by three immunoassays (Roche Elecsys® Anti Spike, Snibe MAGLUMI® S-RBD IgG, and Snibe MAGLUMI® S/N IgG tests), and an ELISA sVNT test (EUROIMMUN® NeutraLISA). The results were compared with the PRNT50 in terms of correlation through Pearson’s and Spearman’s coefficients. Moreover, ROC curves were built to assess the performance of the serological assays to predict NAb, comparing the results with those of the PRNT50 method. Cohen’s Kappa coefficient and agreement percentage were also calculated to study the concordance of the outcomes.

2. Results

The diagnostic performances of three serological immunoassays and an ELISA surrogate viral neutralization test (sVNT) were herein assessed on 83 plasma samples. As a reference, the in vitro neutralization test (PRNT50) was performed, leading to the classification of the antibody titers reported in Table 1. The same samples were tested with each of the four immunoassays. However, the three automated tests (Roche Elecsys® Anti Spike, Snibe MAGLUMI® S-RBD IgG, and Snibe MAGLUMI® S/N IgG tests) were not performed on 10 samples due to the lack of required serum volumes.
The results obtained by serological methods are summarized in Figure 1, while raw data are available in Table S1 (Supplementary Material). For each assay, the scatter plots (Figure 1) displayed the four groups of NAb titers identified by the PRNT50 assay. The Roche Elecsys® Anti Spike test confirmed the presence of anti-SARS-CoV-2 antibodies in all the tested samples. Indeed, no samples falling below the cut-off suggested by the vendor (<0.8 U/mL) were detected (Figure 1a, dotted line). The same applies for the Snibe MAGLUMI® S-RBD IgG test (Figure 1b, dotted line). Conversely, the Snibe MAGLUMI® S/N IgG test classified 12 samples as negative since they fell below the cut-off of 1AU/mL (Figure 1c, dotted line). All the 12 samples belonged to the PRNT50 group with the lowest NAb titer (<80). Similarly, 12 samples were labeled as negative (%HI < 20) by the EUROIMMUN® NeutraLISA assay (Figure 1d, dotted line), 11 of which belong to the NAb titer group < 80, and 1 belongs to the NAb titer group > 80. Furthermore, 10 samples provided a borderline result (%HI between 20 and 35%).
The one-way ANOVA with Bonferroni correction highlighted that the Roche Elecsys® Anti Spike test detected significant differences between samples with NAb titers < 1:80 and the three groups of 1:80, 1:160, and 1:320 (Figure 1a, * p < 0.05). However, among these three groups, no significant differences were observed. Similarly, the Snibe MAGLUMI® S-RBD IgG test was unable to discriminate between NAb titers > 80 and >320, as well as between NAb titers > 160 and >320 (Figure 1b). Concerning the Snibe MAGLUMI® S/N IgG assay, the groups with NAb titers > 80 and >160 were significantly different (* p < 0.05). In addition, the group with an NAb titer < 80 was significantly different from those >160 and >320 (Figure 1c). Finally, the EUROIMMUN® NeutraLISA test was able to distinguish all the groups except those with NAb titers > 160 and >320 (Figure 1d).
Then, in order to compare the reference method to the outcomes of the serological tests, a linear regression analysis was performed (Figure 2). As also shown in Table 2, the calculated Pearson’s coefficients suggested that the relationship between the results of the PRNT50 and those of the EUROIMMUN® NeutraLISA assay, as well as of the Roche Elecsys® Anti Spike test, approached linearity (r = 0.660 and r = 0.617, respectively). Conversely, both the Snibe MAGLUMI® assays displayed a weaker linear correlation with the NAb titers of PRNT50 (r = 0.392 and r = 0.364 for SARS-CoV-2 S-RBD IgG and 2019-nCoV IgG, respectively). Nevertheless, calculating the Spearman’s rank correlation coefficients, a positive monotonic trend was confirmed for all the serological assays (Table 3).
Furthermore, to evaluate the performance of the serological assays to predict NAb titers > 160, ROC curves were built (Figure 3). The cut-off values were chosen by means of Youden’s index (Table 4).
The values of the area under curve (AUC) were >0.8 for each of the four assays. Furthermore, the accuracy was always better than 0.7, as well as sensitivity, while specificity ranged from 0.6 to 0.8 (Table 4). Among the four assays, the EUROIMMUN® NeutraLISA test achieved the best performance, with an AUC of 0.921, indicating good predictive power of the NeutraLISA test. Furthermore, the Youden’s index allowed a maximum accuracy of 0.857, choosing the cut-off % HI equal to 63.3% (Figure 3d, black point). Sensitivity and specificity were both over 0.8 (Table 4). As an alternative, the odds ratio method resulted in a lower cut-off (%HI = 53.5, Figure 3d, white point), maximizing sensitivity in spite of specificity (0.972 and 0.659, respectively), as reported in Table S2. Additionally, the Snibe MAGLUMI® S-RBD IgG test displayed an AUC > 0.9 and reached a sensitivity of 1.000 while its specificity was 0.691 (Table 4). Satisfying results were also obtained for Roche Elecsys® Anti Spike and Snibe MAGLUMI® S/N IgG assays.
Moreover, to explore the concordance between PRNT50 and the outcomes of the serological assays, Cohen’s Kappa coefficients were calculated (Table 5). As expected, the EUROIMMUN® NeutraLISA test achieved the highest Kappa coefficient, followed by the Roche Elecsys® Anti Spike assay. Furthermore, the Snibe MAGLUMI® S/N test had a Kappa coefficient equal to 0.913, while the Snibe MAGLUMI® S-RBD IgG reached a value of only 0.650. According to the interpretation provided by McHugh et al. [21], the concordance was almost perfect for all the assays, except for Snibe MAGLUMI® S-RBD IgG, which showed a moderate concordance. These results followed the same trend observed when calculating the positive predictive agreement for each assay (Table 5).

3. Discussion

In this work, a detailed statistical analysis was performed to compare the outcomes of four different immunoassays (i.e., three serological tests and an ELISA sVNT) with those of the reference method (PRNT50). The ultimate aim of the work is to demonstrate the consistent correlation between an sVNT and the PRNT50 assay. Indeed, a simple and reliable serological test could overcome the requirements of PRNT50 (i.e., specialized laboratories, several days, skilled operators), opening up new landscapes for large-scale epidemiological studies, even in low-resource settings. In this respect, when choosing a test to assess the humoral immunity against SARS-CoV-2, the ASSURED criteria should be taken into account [22]. Indeed, several methods have been applied to study the interactions between SARS-CoV-2 antigens and neutralizing antibodies, but they are often designed for advanced research facilities. As an example, Sun and co-workers used biolayer interferometry to explore the bond between SARS-CoV-2 Spike protein and human ACE2 receptor in vitro, identifying the key viral residues targeted by neutralizing antibodies [23]. In addition, molecular dynamics simulations proved helpful in the prediction of new spike variants’ behaviors and the effectiveness of human neutralizing antibodies [24]. Furthermore, to develop a high-throughput platform to screen neutralizing antibodies against SARS-CoV-2, Fujimoto et al. carried out two-color fluorescence cross-correlation spectroscopy experiments (FCCS), investigating the quantitative interactions between the spike protein and human soluble ACE2 receptor [25]. Conversely, Rusanen and co-workers labeled the viral spike and nucleoprotein antigens and incubated them with serum samples, developing a “mix and read” immunoassay based on Förster resonance energy transfer (FRET) [26]. However, FRET suffered from lower analytical sensitivity as compared to ELISA tests, so it was unable to detect low antibody responses. For these reasons, several research groups focused on simple immunoassays, especially ELISA-based, trying to establish a correlation between them and the reference viral neutralization test [27,28].
In this work, through the linear regression analysis, a good correlation between the PRNT50 and each of the serological assays was observed (Figure 1 and Figure 2). The EUROIMMUN® NeutraLISA test allowed better distinction between the four NAb titer groups identified by the reference method (Table 2 and Table 3). The Spearman’s coefficient confirmed a positive monotone correlation for each studied assay, while the Pearson’s coefficient allowed us to gain insights into the linearity of such a correlation.
Furthermore, considering the FDA recommendation to select hyperimmune plasma with a NAb titer > 160, in this study, the opportunity to discriminate samples above or below this titer was explored through ROC curves (Figure 3). The results, reported in Table 4 and Table 5, indicate the good predictive performance of all the assays thanks to the homogeneity of the examined samples and are in agreement with literature findings [29]. The highest AUC was recorded for the EUROIMMUN® NeutraLISA test, which also provided the best balance of performance in terms of sensitivity and specificity. The cut-offs calculated by the Youden’s index were always higher than those suggested by the vendors, because they aimed at the prediction of NAb titers > 160. Thus, the cut-off selection should be adjusted depending on the assay’s intended use (i.e., seroprevalence studies, rapid selection of candidates eligible as hyperimmune plasma donors).
The goal of predicting NAb titers > 160 was achieved for the patients herein studied, but larger cohort assessments must be performed. The ability to detect different antibody isotypes could be a contributing factor to consider while comparing different assays. In this respect, the results herein obtained are more promising for the EUROIMMUN® NeutraLISA and the Roche Elecsys® Anti Spike tests, which are able to detect all antibody isotypes, while both Snibe MAGLUMI® assays only target IgG molecules.
Furthermore, the emergence of new SARS-CoV-2 variants of concern (VOC) is challenging the knowledge of humoral immunity gained so far. The latest reports underline that the new VOC may elicit an antibody response which could be unmatched by the recombinant RBD molecules present in the immunoassays [30]. Therefore, the detection of NAb targeting non-RBD regions of SARS-CoV-2 spike protein, as well as those targeting non-spike molecules (i.e., the nucleocapsid viral protein), could be useful to assess the protective humoral immunity developed within a population. In this respect, immunoassays such as the Snibe MAGLUMI® S/N IgG assay should provide better results as compared with the Snibe MAGLUMI® S-RBD IgG test. Moreover, functional assays (e.g., sVNT) should be less biased by arising VOC than traditional immunoassays. For the same reasons, hemagglutination tests (HAT) could be easily adjusted to face the issue of VOC [31]. However, HATs are not certified for diagnostics, even if they are considered helpful tools in low-resource settings [32,33].
Considering the updated literature findings and the results herein described, it can be concluded that the EUROIMMUN® NeutraLISA assay, exploring the interaction between the soluble ACE-2 receptor and the patients’ antibodies, could provide functional evidence of NAb presence without requiring a BSL-3 laboratory setting. Indeed, even if the in vitro viral neutralization assay remains the reference method, we demonstrated that the EUROIMMUN® NeutraLISA assay is consistent with the PRNT50 assay while having the advantages of being quicker and easier to perform. Indeed, the ELISA setup allows the shortening of testing times while significantly reducing costs as well as the need for specifically skilled operators.

4. Materials and Methods

4.1. Samples Collection

Serum samples were collected from SARS-CoV-2-infected patients recruited at the Immunohematology and Transfusion Medicine Service. Samples (n = 83, 14 females and 69 males) were selected after a negative molecular test (COBAS® Roche RT-qPCR assay on nasopharyngeal swab) was performed at least 14 days before enrolment. The patients’ median age was equal to 42 years (IQR 31–51). Each sample was immediately processed and stored at −20 °C until tested with the five methods described below, unless otherwise specified (Table S1, Supplementary Material). It cannot be excluded that gender bias might have a relevant impact on the results herein shown.

4.2. Serological Tests

4.2.1. Roche Elecsys® Anti Spike Test

According to the manufacturer’s instructions (Roche Diagnostics S.p.A, Monza, Italy), the Roche Elecsys® Anti Spike immunoassay was performed on the COBAS® e411 analyzer (Roche Diagnostics S.p.A, Monza, Italy), building a two-point calibration. The test consisted of an Electro-Chemi-Luminescence Indirect Assay (ECLIA) and included two recombinant RBD antigens, which bound the samples’ antibodies in a double-antigen sandwich setup. Briefly, one recombinant RBD antigen was ruthenylated, while the other one was biotinylated. After 9 min of incubation between the two recombinant antigens and the patient serum, a solid phase was added (i.e., microparticles coated with streptavidin) to ease the magnetic recovery of the double antigen–antibody complexes, taking advantage of the biotin-streptavidin bond. After another 9 min of incubation and a washing step, the complexes reached a measuring chamber, in which a voltage was applied. Thus, electrochemiluminescence was triggered and detected by a photomultiplier. The recorded signal increased directly with the number of antibodies within the sample. The measuring range included values between 0.40 and 250 U/mL, but automated sample dilutions (1:10 and 1:100) extended the upper range of quantification to 25,000 U/mL [34].

4.2.2. Snibe MAGLUMI® SARS-CoV-2 S-RBD IgG Test

The indirect chemiluminescence immunoassay (CLIA) was performed by the MAGLUMI® 800 auto-analyzer according to the manufacturer’s instructions, i.e., the Shenzhen New Industries Biomedical Engineering Co., Ltd. (Snibe, Shenzhen, China) [35]. Briefly, the samples were diluted and incubated under vigorous stirring, with magnetic microspheres coated with the recombinant RBD region of SARS-CoV-2 spike protein. After incubation, the antigen–antibody complexes were attracted by a magnetic field to allow the washing of non-bonded molecules. A second incubation was performed with anti-human IgG antibodies labeled with N-(4-Aminobutyl)-N-ethylisoluminol (ABEI). During incubation, labeled antibodies recognized and bound the samples’ IgG. Magnetic attraction allowed us to precipitate the complexes and perform a second wash-out step. Finally, the starter was provided, triggering a chemiluminescent reaction. A photomultiplier detected the resulting signal as relative light units (RLUs) and the autoanalyzer, after proper calibration, was able to calculate the concentration of IgG within the samples.

4.2.3. Snibe MAGLUMI® 2019-nCoV IgG Test

The Snibe MAGLUMI® S/N IgG test was performed as recommended by Snibe (Shenzhen, China), similarly to the test described above. The only difference was the antigens coating the magnetic microparticles, which belonged to the nucleocapsid and the spike proteins [36].

4.2.4. EUROIMMUN® NeutraLISA Test

The test was manually performed according to the supplier (EUROIMMUN Medizinische Labordiagnostika, Lubeck, Germany). The ELISA assay was carried out in microplate strips, in which the reaction wells were coated with the recombinant SARS-CoV-2 S1/RBD domain, obtained through in vitro culture of HEK-293 cells (Human Embryo Kidney, ATCC Number CRL-1573). The patients’ sera were diluted in the supplied sample buffer, which included a biotinylated, soluble human ACE2 receptor. The latter, being the natural ligand of the RBD spike protein, competed with the NAb of the sample. After 1h of incubation, unbound ACE2 molecules were washed away, and a colorimetric reaction was triggered by the addition of the peroxidase enzyme, labeled with streptavidin. Therefore, the color intensity measured at 450nm was inversely correlated to the concentration of NAb within the sample. Results were expressed as inhibition percentage (%IH), calculated according to Equation (1):
%IH = 100% − (Sample Absorbance × 100%/Blank Absorbance)
For %IH between 20 and 35%, the test result was declared borderline, as recommended by the vendor [37].

4.3. Plaque-Reduction Neutralization Test (PRNT50)

The PRNT50 was performed at the BioSafety Level 3 laboratories of the Istituto Zooprofilattico Sperimentale as a reference method to assess the patients’ sera neutralization capacity against SARS-CoV-2. According to Wölfel et al. [38], the patients’ sera were pre-treated for 30 min at 56 °C (heat inactivation) and subsequently tested in duplicate. After serial dilutions in OptiPro™ medium (Thermo Fisher Scientific Inc, Milan, Italy), patients’ sera were incubated for 1h at 37 °C with a 100 PFU solution of SARS-CoV-2 (GISAID ID: EPI_ISL 406862). Meanwhile, Vero-E6 cells (African Green Monkey kidney cells, American Type Culture Collection ID: CRL-1587, Manassas, VA, USA) were seeded on 24-well plates at a concentration of 4 × 105 cells/mL and cultured at 37 °C, 5% CO2, overnight. The preincubated SARS-CoV-2 viral solutions were added to the Vero-E6-seeded wells, incubating them for 1h at 37 °C. Then, the supernatant was discarded, and a PBS wash was performed. The cultures were supplied with DMEM and 1.2% (w/v) Avicel® microcrystalline cellulose solution (Sigma Aldrich, Milan, Italy) and left for 3 days at 37°C. After that, the supernatant was discarded, and the cells were fixed in a 6% v/v formaldehyde solution in PBS for 30 min. Finally, the cell monolayers were washed twice with PBS, stained with a 0.2% w/v of crystal violet solution, rinsed with PBS, and dried for plaque counting. Neutralizing antibody (NAb) titers were calculated as the corresponding serum dilution eliciting a plaque reduction equal to 50% [39].

4.4. Statistical Analyses

Data analysis was performed by R software (version 4.0.1., R Development Core Team), loading the packages ggplot2, ROCR, cutpointr, pROC, and vcd [40,41,42,43,44]. The level of statistical significance was set at p < 0.05. The Tukey method was used to identify and remove outliers. One-way ANOVA, followed by Bonferroni correction for multiple comparisons, was performed to study the results of the four serological tests. Linear regression and Spearman coefficient analyses allowed us to correlate the outcomes of the serological methods tested as compared to PRNT50. Furthermore, ROC curves were built to assess the ability of a serological test to identify a neutralizing antibody titer ≥ 1:160. The cut-off was calculated using the Youden’s Index and the odds ratio, while Cohen’s Kappa coefficient and positive predictive agreement were calculated to assess the concordance of the obtained results.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/ijms23179566/s1.

Author Contributions

Conceptualization, M.A.M., M.A.B. and R.L.; methodology, investigation, R.L., A.V., A.S. and A.F.; formal analysis, data curation, M.A.B., R.L., F.D.S. and A.O.; writing—original draft preparation, M.A.B., R.L., M.D.; writing—review and editing, M.A.M. and M.A.B.; supervision, M.A.M. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Ethical review and approval were waived for this study, considering its retrospective and observational nature.

Informed Consent Statement

This research was carried out in agreement with Italian and institutional standards, as well as in accordance with the guidelines of the Declaration of Helsinki, revised in 2013. All data herein presented were collected during diagnostic routine at the University Hospital Policlinico, Bari.

Data Availability Statement

All reported data are herein available. Raw data are available from the corresponding author, on reasonable request.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Stadlbauer, D.; Amanat, F.; Chromikova, V.; Jiang, K.; Strohmeier, S.; Arunkumar, G.A.; Tan, J.; Bhavsar, D.; Capuano, C.; Kirkpatrick, E.; et al. SARS-CoV-2 seroconversion in humans: A detailed protocol for a serological assay, antigen production, and test setup. Curr. Protoc. Microbiol. 2020, 57, e100. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  2. Kubina, R.; Dziedzic, A. Molecular and serological tests for COVID-19. A comparative review of SARS-CoV-2 coronavirus laboratory and point-of-care diagnostics. Diagnostics 2020, 10, 434. [Google Scholar] [CrossRef] [PubMed]
  3. Kumar, G.; Sterrett, S.; Hall, L.; Tabengwa, E.; Honjo, K.; Larimer, M.; Davis, R.S.; Goepfert, P.A.; Larimer, B.M. Comprehensive mapping of SARS-CoV-2 peptide epitopes for development of a highly sensitive serological test for total and neutralizing antibodies. Protein Eng. Des. Sel. 2022, 35, gzab033. [Google Scholar] [CrossRef] [PubMed]
  4. Ernst, E.; Wolfe, P.; Stahura, C.; Edwards, K.A. Technical considerations to development of serological tests for SARS-CoV-2. Talanta 2021, 224, 121883. [Google Scholar] [CrossRef] [PubMed]
  5. Bond, K.; Nicholson, S.; Lim, S.M.; Karapanagiotidis, T.; Williams, E.; Johnson, D.; Hoang, T.; Sia, C.; Purcell, D.; Mordant, F.; et al. Evaluation of serological tests for SARS-CoV-2: Implications for serology testing in a low-prevalence setting. J. Infect. Dis. 2020, 222, 1280–1288. [Google Scholar] [CrossRef]
  6. Wu, X.; Fu, B.; Chen, L.; Feng, Y. Serological tests facilitate identification of asymptomatic SARS-CoV-2 infection in Wuhan, China. J. Med. Virol. 2020. [Google Scholar] [CrossRef] [Green Version]
  7. European Centre for Disease Prevention and Control. Considerations for the Use of Antibody Tests for SARSCoV-2–First Update; ECDC: Stockholm, Sweden, 2022.
  8. Milne, G.; Hames, T.; Scotton, C.; Gent, N.; Johnsen, A.; Anderson, R.M.; Ward, T. Does infection with or vaccination against SARS-CoV-2 lead to lasting immunity? Lancet Respir. Med. 2021, 9, 1450–1466. [Google Scholar] [CrossRef]
  9. Jordan, S.C. Innate and adaptive immune responses to SARS-CoV-2 in humans: Relevance to acquired immunity and vaccine responses. Clin. Exp. Immunol. 2021, 204, 310–320. [Google Scholar] [CrossRef]
  10. Galipeau, Y.; Greig, M.; Liu, G.; Driedger, M.; Langlois, M.A. Humoral responses and serological assays in SARS-CoV-2 infections. Front. Immunol. 2020, 11, 610688. [Google Scholar] [CrossRef]
  11. Cromer, D.; Steain, M.; Reynaldi, A.; Schlub, T.E.; Wheatley, A.K.; Juno, J.A.; Kent, S.J.; A Triccas, J.; Khoury, D.S.; Davenport, M.P. Neutralising antibody titres as predictors of protection against SARS-CoV-2 variants and the impact of boosting: A meta-analysis. Lancet Microbe 2022, 3, e52–e61. [Google Scholar] [CrossRef]
  12. Laterza, R.; Schirinzi, A.; Bruno, R.; Genco, R.; Contino, R.; Ostuni, A.; Di Serio, F. SARS-CoV-2 antibodies: Comparison of three high-throughput immunoassays versus the neutralization test. Eur. J. Clin. Investig. 2021, 51, e13573. [Google Scholar] [CrossRef] [PubMed]
  13. Bewley, K.R.; Coombes, N.S.; Gagnon, L.; McInroy, L.; Baker, N.; Shaik, I.; St-Jean, J.R.; St-Amant, N.; Buttigieg, K.R.; Humphries, H.E.; et al. Quantification of SARS-CoV-2 neutralizing antibody by wild-type plaque reduction neutralization, microneutralization and pseudotyped virus neutralization assays. Nat. Protoc. 2021, 16, 3114–3140. [Google Scholar] [CrossRef] [PubMed]
  14. Xie, X.; Nielsen, M.C.; Muruato, A.E.; Fontes-Garfias, C.R.; Ren, P. Evaluation of a SARS-CoV-2 lateral flow assay using the plaque reduction neutralization test. Diagn. Microbiol. Infect. Dis. 2021, 99, 115248. [Google Scholar] [CrossRef]
  15. Liu, J.; Liu, Y.; Xia, H.; Zou, J.; Weaver, S.C.; Swanson, K.A.; Cai, H.; Cutler, M.; Cooper, D.; Muik, A.; et al. BNT162b2-elicited neutralization of B. 1.617 and other SARS-CoV-2 variants. Nature 2021, 596, 273–275. [Google Scholar] [CrossRef] [PubMed]
  16. Tan, C.W.; Chia, W.N.; Qin, X.; Liu, P.; Chen, M.I.C.; Tiu, C.; Hu, Z.; Chih-Wei Chen, V.; Young, B.E.; Sia, W.R.; et al. A SARS-CoV-2 surrogate virus neutralization test based on antibody-mediated blockage of ACE2–spike protein–protein interaction. Nat. Biotechnol. 2020, 38, 1073–1078. [Google Scholar] [CrossRef]
  17. Lynch, K.L.; Zhou, S.; Kaul, R.; Walker, R.; Wu, A.H. Evaluation of neutralizing antibodies against SARS-CoV-2 variants after infection and vaccination using a multiplexed surrogate virus neutralization test. Clin. Chem. 2022, 68, 702–712. [Google Scholar] [CrossRef] [PubMed]
  18. Suhandynata, R.T.; Hoffman, M.A.; Huang, D.; Tran, J.T.; Kelner, M.J.; Reed, S.L.; McLawhon, R.W.; Voss, J.E.; Nemazee, D.; Fitzgerald, R.L. Commercial serology assays predict neutralization activity against SARS-CoV-2. Clin. Chem. 2021, 67, 404–414. [Google Scholar] [CrossRef]
  19. Ong, D.S.; Keuren, F.; van der Vliet, M.; Boxma–de Klerk, B.M.; Koeleman, J.G. SARS-CoV-2 antibody response dynamics and heterogeneous diagnostic performance of four serological tests and a neutralization test in symptomatic healthcare workers with non-severe COVID-19. J. Clin. Virol. 2021, 141, 104904. [Google Scholar] [CrossRef]
  20. Rowntree, L.C.; Chua, B.Y.; Nicholson, S.; Koutsakos, M.; Hensen, L.; Douros, C.; Selva, K.; Mordant, F.L.; Wong, C.Y.; Habel, J.R.; et al. Robust correlations across six SARS-CoV-2 serology assays detecting distinct antibody features. Clin. Transl. Immunol. 2021, 10, e1258. [Google Scholar] [CrossRef]
  21. McHugh, M.L. Interrater reliability: The kappa statistic. Biochem. Med. 2012, 22, 276–282. [Google Scholar] [CrossRef]
  22. Mabey, D.; Peeling, R.W.; Ustianowski, A.; Perkins, M.D. Diagnostics for the developing world. Nat. Rev. Microbiol. 2004, 2, 231–240. [Google Scholar] [CrossRef] [PubMed]
  23. Yi, C.; Sun, X.; Ye, J.; Ding, L.; Liu, M.; Yang, Z.; Lu, X.; Zhang, Y.; Ma, L.; Gu, W.; et al. Key residues of the receptor binding motif in the spike protein of SARS-CoV-2 that interact with ACE2 and neutralizing antibodies. Cell Mol. Immunol. 2020, 17, 621–630. [Google Scholar] [CrossRef] [PubMed]
  24. Tragni, V.; Preziusi, F.; Laera, L.; Onofrio, A.; Mercurio, I.; Todisco, S.; Volpicella, M.; De Grassi, A.; Pierri, C.L. Modeling SARS-CoV-2 spike/ACE2 protein–protein interactions for predicting the binding affinity of new spike variants for ACE2, and novel ACE2 structurally related human protein targets, for COVID-19 handling in the 3PM context. EPMA J. 2022, 13, 149–175. [Google Scholar] [CrossRef]
  25. Fujimoto, A.; Lyu, Y.; Kinjo, M.; Kitamura, A. Interaction between Spike Protein of SARS-CoV-2 and Human Virus Receptor ACE2 Using Two-Color Fluorescence Cross-Correlation Spectroscopy. Appl. Sci. 2021, 11, 10697. [Google Scholar] [CrossRef]
  26. Rusanen, J.; Kareinen, L.; Levanov, L.; Mero, S.; Pakkanen, S.H.; Kantele, A.; Amanat, F.; Krammer, F.; Hedman, K.; Vapalahti, O.; et al. A 10-Minute “Mix and Read” Antibody Assay for SARS-CoV-2. Viruses 2021, 13, 143. [Google Scholar] [CrossRef]
  27. Harvala, H.; Robb, M.L.; Watkins, N.; Ijaz, S.; Dicks, S.; Patel, M.; Supasa, P.; Wanwisa, D.; Liu, C.; Mongkolsapaya, J.; et al. Convalescent plasma therapy for the treatment of patients with COVID-19: Assessment of methods available for antibody detection and their correlation with neutralising antibody levels. Transfus. Med. 2021, 31, 167–175. [Google Scholar] [CrossRef] [PubMed]
  28. Girl, P.; Zwirglmaier, K.; von Buttlar, H.; Wölfel, R.; Müller, K. Evaluation of Two Rapid Lateral Flow Tests and Two Surrogate ELISAs for the Detection of SARS-CoV-2 Specific Neutralizing Antibodies. Front. Med. 2022, 9, 820151. [Google Scholar] [CrossRef] [PubMed]
  29. Bal, A.; Pozzetto, B.; Trabaud, M.A.; Escuret, V.; Rabilloud, M.; Langlois-Jacques, C.; Paul, A.; Guibert, N.; D’Aubarède-Frieh, C.; Massardier-Pilonchery, A.; et al. COVID SER Study Group. Evaluation of high-throughput SARS-CoV-2 serological assays in a longitudinal cohort of patients with mild COVID-19: Clinical sensitivity, specificity, and association with virus neutralization test. Clin. Chem. 2021, 67, 742–752. [Google Scholar] [CrossRef]
  30. Saker, K.; Pozzetto, B.; Bal, A.; Valette, M.; Fassier, J.B.; Saade, C.; Trabaud, M.A.; Trouillet-Assant, S.; COVID SER Study Group. Are Anti-Receptor–Binding Domain Antibodies Still a Relevant Proxy for Monitoring SARS-CoV-2 Neutralizing Activity in the Omicron Era? Clin. Chem. 2022, 68, 984–986. [Google Scholar] [CrossRef]
  31. Townsend, A.; Rijal, P.; Xiao, J.; Tan, T.K.; Huang, K.-Y.A.; Schimanski, L.; Huo, J.; Gupta, N.; Rahikainen, R.; Matthews, P.C.; et al. A haemagglutination test for rapid detection of antibodies to SARS-CoV-2. Nat. Commun. 2021, 12, 1951. [Google Scholar] [CrossRef]
  32. Kruse, R.L.; Huang, Y.; Lee, A.; Zhu, X.; Shrestha, R.; Laeyendecker, O.; Littlefield, K.; Pekosz, A.; Bloch, E.M.; Tobian, A.A.R.; et al. A Hemagglutination-Based Semiquantitative Test for Point-of-Care Determination of SARS-CoV-2 Antibody Levels. J. Clin. Microbiol. 2021, 59, 12. [Google Scholar] [CrossRef] [PubMed]
  33. Focosi, D.; Franchini, M.; Maggi, F. Modified Hemagglutination Tests for COVID-19 Serology in Resource-Poor Settings: Ready for Prime-Time? Vaccines 2022, 10, 406. [Google Scholar] [CrossRef] [PubMed]
  34. Jochum, S.; Kirste, I.; Hortsch, S.; Grunert, V.P.; Legault, H.; Eichenlaub, U.; Kashlan, B.; Pajon, R. Clinical utility of Elecsys Anti-SARS-CoV-2 S assay in COVID-19 vaccination: An exploratory analysis of the mRNA-1273 phase 1 trial. Front. Immunol. 2021, 12. [Google Scholar] [CrossRef] [PubMed]
  35. Soleimani, R.; Khourssaji, M.; Gruson, D.; Rodriguez-Villalobos, H.; Berghmans, M.; Belkhir, L.; Yombi, J.; Kabamba-Mukadi, B. Clinical usefulness of fully automated chemiluminescent immunoassay for quantitative antibody measurements in COVID-19 patients. J. Med. Virol. 2021, 93, 1465–1477. [Google Scholar] [CrossRef]
  36. Lippi, G.; Salvagno, G.L.; Pegoraro, M.; Militello, V.; Caloi, C.; Peretti, A.; Gaino, S.; Bassi, A.; Bovo, C.; Cascio, G.L. Assessment of immune response to SARS-CoV-2 with fully automated MAGLUMI 2019-nCoV IgG and IgM chemiluminescence immunoassays. Clin. Chem. Lab. Med. 2020, 58, 1156–1159. [Google Scholar] [CrossRef] [Green Version]
  37. Hofmann, N.; Grossegesse, M.; Neumann, M.; Schaade, L.; Nitsche, A. Evaluation of a commercial ELISA as alternative to plaque reduction neutralization test to detect neutralizing antibodies against SARS-CoV-2. Sci. Rep. 2022, 12, 1–9. [Google Scholar] [CrossRef]
  38. Wölfel, R.; Corman, V.M.; Guggemos, W.; Seilmaier, M.; Zange, S.; Müller, M.A.; Niemeyer, D.; Jones, T.C.; Vollmar, P.; Rothe, C.; et al. Virological assessment of hospitalized patients with COVID-2019. Nature 2020, 581, 465–469. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  39. Wang, S.; Sakhatskyy, P.; Chou, T.H.W.; Lu, S. Assays for the assessment of neutralizing antibody activities against Severe Acute Respiratory Syndrome (SARS) associated coronavirus (SCV). J. Immunol. Methods 2005, 301, 21–30. [Google Scholar] [CrossRef] [PubMed]
  40. Wickham, H. Ggplot2: Elegant Graphics for Data Analysis, 2nd ed.; Springer: New York, NY, USA, 2016; ISBN 978-3-319-24277-4. [Google Scholar]
  41. Sing, T.; Sander, O.; Beerenwinkel, N.; Lengauer, T. ROCR: Visualizing classifier performance in R. Bioinformatics 2005, 21, 3940–3941. [Google Scholar] [CrossRef]
  42. Thiele, C.; Hirschfeld, G. The cutpointr package: Improved and tidy estimation of optimal cutpoints. In Proceedings of the Statistical Computing 2017, Ulmer Informatik-Berichte, Günzburg, Germany, 23–25 July 2017. [Google Scholar]
  43. Robin, X.; Turck, N.; Hainard, A.; Tiberti, N.; Lisacek, F.; Sanchez, J.C.; Müller, M. pROC: An open-source package for R and S+ to analyze and compare ROC curves. BMC Bioinform. 2011, 12, 77. [Google Scholar] [CrossRef]
  44. Meyer, D.; Zeileis, A.; Hornik, K. The strucplot framework: Visualizing multi-way contingency tables with vcd. J. Stat. Softw. 2007, 17, 1–48. [Google Scholar] [CrossRef]
Ijms 23 09566 g001 550
Figure 1. Scatter plots showing the results of the four serological tests, grouped according to PRNT50 NAb titers. Solid lines represent median values, white rhombi mean values. (a) Roche Elecsys® Anti Spike test (cut off ≥ 0.80 U/mL), (b) Snibe MAGLUMI® S-RBD IgG test (cut off ≥ 1.00 AU/mL), (c) Snibe MAGLUMI® S/N IgG test (cut off ≥ 1.00 AU/mL), (d) EUROIMMUN® NeutraLISA assay (cut off <20% for negative test and up to 35% for borderline values). Dotted lines represent the cut-off for each test. Statistically significant differences between NAb titer groups were assessed by one-way ANOVA and Bonferroni correction (* p < 0.05. ns: not significant).
Figure 1. Scatter plots showing the results of the four serological tests, grouped according to PRNT50 NAb titers. Solid lines represent median values, white rhombi mean values. (a) Roche Elecsys® Anti Spike test (cut off ≥ 0.80 U/mL), (b) Snibe MAGLUMI® S-RBD IgG test (cut off ≥ 1.00 AU/mL), (c) Snibe MAGLUMI® S/N IgG test (cut off ≥ 1.00 AU/mL), (d) EUROIMMUN® NeutraLISA assay (cut off <20% for negative test and up to 35% for borderline values). Dotted lines represent the cut-off for each test. Statistically significant differences between NAb titer groups were assessed by one-way ANOVA and Bonferroni correction (* p < 0.05. ns: not significant).
Ijms 23 09566 g001
Ijms 23 09566 g002 550
Figure 2. Linear regression analyses related to (a) Roche Elecsys® Anti Spike test, (b) Snibe MAGLUMI® S-RBD IgG test (c) Snibe MAGLUMI® S/N IgG test, (d) EUROIMMUN® NeutraLISA assay. Grey areas represent confidence intervals; black dotted lines represent the linear model.
Figure 2. Linear regression analyses related to (a) Roche Elecsys® Anti Spike test, (b) Snibe MAGLUMI® S-RBD IgG test (c) Snibe MAGLUMI® S/N IgG test, (d) EUROIMMUN® NeutraLISA assay. Grey areas represent confidence intervals; black dotted lines represent the linear model.
Ijms 23 09566 g002
Ijms 23 09566 g003 550
Figure 3. ROC curves relevant to (a) Roche Elecsys® Anti Spike test, (b) Snibe MAGLUMI® S-RBD IgG test, (c) Snibe MAGLUMI® S/N IgG test and (d) EUROIMMUN® NeutraLISA test. The black points indicate Youden’s index-optimized cut-off, and the white points indicate the odds-ratio optimized cut-off.
Figure 3. ROC curves relevant to (a) Roche Elecsys® Anti Spike test, (b) Snibe MAGLUMI® S-RBD IgG test, (c) Snibe MAGLUMI® S/N IgG test and (d) EUROIMMUN® NeutraLISA test. The black points indicate Youden’s index-optimized cut-off, and the white points indicate the odds-ratio optimized cut-off.
Ijms 23 09566 g003
Table
Table 1. Results of the PRNT50 assay, performed on 83 patients, after healing from severe SARS-CoV-2 infection.
Table 1. Results of the PRNT50 assay, performed on 83 patients, after healing from severe SARS-CoV-2 infection.
Neutralizing Antibody TitersNumber of Samples
<8025
>8020
>16021
>32017
Table
Table 2. Results of the linear regression modeling. r: Pearson’s coefficient; β0: slope; β1: intercept.
Table 2. Results of the linear regression modeling. r: Pearson’s coefficient; β0: slope; β1: intercept.
TestrR2p-Valueβ0β1
Roche Elecsys® Anti Spike test0.6170.3811 × 10−80.12990.735
Snibe MAGLUMI® S-RBD IgG test0.3920.1531 × 10−30.36595.386
Snibe MAGLUMI® S/N IgG test0.3640.1323 × 10−37.19295.383
EUROIMMUN® NeutraLISA assay0.6600.4369 × 10−111.95925.044
Table
Table 3. Spearman’s rank correlation coefficients (r) relevant to PRNT50 results and each of the studied serological assays.
Table 3. Spearman’s rank correlation coefficients (r) relevant to PRNT50 results and each of the studied serological assays.
Testrp-Value
Roche Elecsys® Anti Spike test0.7268 × 10−13
Snibe MAGLUMI® S-RBD IgG test0.7308 × 10−13
Snibe MAGLUMI® S/N IgG test0.6816 × 10−10
EUROIMMUN® NeutraLISA assay0.7842 × 10−16
Table
Table 4. Optimized parameters from the ROC curves, calculated by the Youden’s index, relevant to each serological assay.
Table 4. Optimized parameters from the ROC curves, calculated by the Youden’s index, relevant to each serological assay.
TestAUCCut-offAccuracySensitivitySpecificity
Roche Elecsys® Anti Spike test0.843210 U/mL0.7940.7330.837
Snibe MAGLUMI® S-RBD IgG test0.91632.7AU/mL0.8191.0000.691
Snibe MAGLUMI® S/N IgG test0.8022.9 AU/mL0.7640.9670.619
EUROIMMUN® NeutraLISA assay0.92163.3%0.8570.8890.829
Table
Table 5. Analyses of concordance relevant to each serological assay.
Table 5. Analyses of concordance relevant to each serological assay.
TestCohen’s Kappa Coefficient (CI)Positive Predictive Agreement
Roche Elecsys® Anti Spike test0.972 (0.916–1.000)96.7%
Snibe MAGLUMI® S-RBD IgG test0.650 (0.489–0.811)69.8%
Snibe MAGLUMI® S/N IgG test0.913 (0.817–1.000)93.3%
EUROIMMUN® NeutraLISA assay0.975 (0.925–1.000)97.4%
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Bonifacio, M.A.; Laterza, R.; Vinella, A.; Schirinzi, A.; Defilippis, M.; Di Serio, F.; Ostuni, A.; Fasanella, A.; Mariggiò, M.A. Correlation between In Vitro Neutralization Assay and Serological Tests for Protective Antibodies Detection. Int. J. Mol. Sci. 2022, 23, 9566. https://doi.org/10.3390/ijms23179566

AMA Style

Bonifacio MA, Laterza R, Vinella A, Schirinzi A, Defilippis M, Di Serio F, Ostuni A, Fasanella A, Mariggiò MA. Correlation between In Vitro Neutralization Assay and Serological Tests for Protective Antibodies Detection. International Journal of Molecular Sciences. 2022; 23(17):9566. https://doi.org/10.3390/ijms23179566

Chicago/Turabian Style

Bonifacio, Maria Addolorata, Riccardo Laterza, Angela Vinella, Annalisa Schirinzi, Mariangela Defilippis, Francesca Di Serio, Angelo Ostuni, Antonio Fasanella, and Maria Addolorata Mariggiò. 2022. "Correlation between In Vitro Neutralization Assay and Serological Tests for Protective Antibodies Detection" International Journal of Molecular Sciences 23, no. 17: 9566. https://doi.org/10.3390/ijms23179566

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop