Next Article in Journal
Advanced Imaging Analysis in Prostate MRI: Building a Radiomic Signature to Predict Tumor Aggressiveness
Previous Article in Journal
Visualizing Glioma Infiltration by the Combination of Multimodality Imaging and Artificial Intelligence, a Systematic Review of the Literature
 
 
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Five Commercial Immunoassays for SARS-CoV-2 Antibody Determination and Their Comparison and Correlation with the Virus Neutralization Test

1
Department of Immunochemistry Diagnostics, University Hospital in Pilsen, 309 55 Pilsen, Czech Republic
2
Laboratory of Immunology, Gennet, 110 00 Prague, Czech Republic
3
Faculty of Science, Palacky University in Olomouc, 779 00 Olomouc, Czech Republic
4
Department of Microbiology, University Hospital in Pilsen, 309 55 Pilsen, Czech Republic
5
Department of Infectious Diseases and Travel Medicine, University Hospital in Pilsen, 309 55 Pilsen, Czech Republic
6
Institute of Epidemiology, Faculty of Medicine in Pilsen, Charles University, 305 99 Pilsen, Czech Republic
7
Department of Virology, Public Health Institute, 702 00 Ostrava, Czech Republic
8
Department of Biomedical Sciences, Faculty of Medicine, University of Ostrava, 703 00 Ostrava, Czech Republic
9
Department of Otorhinolaryngology, University Hospital in Pilsen, Faculty of Medicine in Pilsen, Charles University, 305 99 Pilsen, Czech Republic
10
Institute of Pharmacology and Toxicology, Faculty of Medicine in Pilsen, Charles University, 323 00 Pilsen, Czech Republic
*
Author to whom correspondence should be addressed.
Diagnostics 2021, 11(4), 593; https://doi.org/10.3390/diagnostics11040593
Received: 4 March 2021 / Revised: 19 March 2021 / Accepted: 21 March 2021 / Published: 25 March 2021
(This article belongs to the Section Diagnostic Microbiology and Infectious Disease)

Abstract

:
There is an ongoing debate as to whether SARS-CoV-2 antibodies can be found in patients who have recovered from COVID-19 disease. Currently, there is no consensus on whether the antibodies, if present, are protective. Our regular measurements of SARS-CoV-2 antibodies, starting in July 2020, have provided us with the opportunity of becoming acquainted with the five different immunoassays. A total of 149 patients were enrolled in our study. We measured the samples using each immunoassay, then performing a virus neutralization test and comparing the results of SARS-CoV-2 antibodies with this test. We observed that the production of neutralizing antibodies is age-dependent. Elderly patients have a higher proportion of high neutralizing titers than young patients. Based on our results, and in combination with the literature findings, we can conclude that the serological SARS-CoV-2 antibody measurement is a helpful tool in the fight against COVID-19. The assays can provide information about the patient’s previous contact with the virus. Anti-spike protein assays correlate well with the virus neutralization test and can be used in the screening of potential convalescent plasma donors.

1. Introduction

Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) which causes the coronavirus disease 2019 (COVID-19), was first detected in Wuhan, China, in November 2019 [1]. Thus far, the SARS-CoV-2 virus has infected over 100 million individuals with 2.5 million deaths globally and has caused a pandemic which has affected almost every country worldwide [2]. There is an ongoing debate as to whether SARS-CoV-2 antibodies can be found in patients who have recovered from COVID-19 disease. Currently, there is no consensus on whether the antibodies, if present, are protective [3]. Detecting antibodies allows us to confirm that the person was in contact with the virus and developed an immune response against the coronavirus infection. Such a person has a higher likelihood of being protected against a future infection. After the disease, the immune system has a large spectrum of weapons ready for a repeated contact with the virus—especially the specific T cell and humoral immunity [4,5]. Clinical experience gained from studying patients who have been reinfected with other types of coronaviruses indicated that in the 6–12 months following the first infection, the viral load was lower, the period of virus shedding was shorter than in the previous infection, the course of the disease was much weaker, or the disease did not develop [6,7].
Test kits for the presence of antibodies against SARS-CoV-2 gradually began to appear on the market in the spring of 2020. Depending on the type of antigen, their sensitivity and specificity varied [8]. Antibodies against SARS-CoV-2 can target several proteins. Currently, nucleocapsid proteins and spike proteins are most commonly used as the targets for serological assays. It is known from other coronaviruses that the spike protein is the main inducer of neutralizing antibodies [9]. Virus neutralizing antibodies protect the organism by blocking the interaction between the virus and the potentially susceptible host cells.
For a long time, the virus neutralization test (VNT) has been routinely used as a functional test for the assessment of the neutralizing ability of the antibodies produced in the organism [10]. COVID-19 convalescent plasma administration is a type of passive antibody therapy and is used as an experimental treatment option in many countries worldwide [11]. The patients can benefit from the high titer of the virus neutralizing antibodies which can neutralize the virus and help recipients defeat the virus [12].
The aim of this study was to compare five different immunoassays for the determination of SARS-CoV-2 antibodies. First, each assay was compared to the remaining four, and those were then compared to the VNT in order to assess the method’s ability to predict the level of antibody neutralization which could allow the screening of potential plasma donors. We also studied the dependence of the test results on age.

2. Material and Methods

2.1. Group of Patients

SARS-CoV-2 antibodies were measured between July 24 and August 27, 2020, in 149 patients (55 males, 94 females). All the patients enrolled in the study were subjects from the Pilsen region with a confirmed COVID-19 diagnosis, all with mild clinical symptoms. All diagnostic tests and examinations were performed in the laboratories of University Hospital in Pilsen. The diagnosis was established based on a positive PCR test and/or the presence of clinical symptoms and/or a positive finding using an imaging technique. The blood samples were collected between days 35 and 48 following a positive PCR test result, or the day following a negative PCR result where clinical symptoms were present. It was assumed that by that point the level of IgG antibodies in the patients who were able to produce them would already be detectable. The clinical characteristics of the patient group are shown in Table 1. Informed consent was obtained from all patients enrolled to the study. The study was approved by the Ethics Committee of University Hospital and Medical Faculty in Pilsen on 8 August 2020 (approval number 354/2020).

2.2. Serum Samples

Samples of peripheral venous blood were collected using the VACUETTE blood collection system (Greiner Bio-one Company, Kremsmünster, Austria). Serum was separated by 10 minute centrifugation at 1300× g, aliquoted into three 500 µL aliquots, and frozen at −80 °C. One aliquot was used for SARS-CoV-2 antibody determination, a second aliquot was used for VNT, and a third aliquot was used as back-up. Aliquots were thawed only once, prior to the immunochemical analyses or VNT.

2.3. Immunoassays

Five commercial assays were used in the study. Four of them were chemiluminescent (CLIA) automated assays, and one was an enzyme-linked immunosorbent assay (ELISA) manual assay. We used a CLIA ELECSYS Anti-SARS-CoV-2 assay for the detection of the total-immunoglobulin (total Ig: IgM, IgG, and IgA) antibodies against the nucleocapsid protein (NP) (Roche, Basel, Switzerland); CLIA ARCHITECT SARS-CoV-2 IgG for detection of the IgG antibodies against the nucleocapsid protein (IgG NP) (Abbott, Libertyville, IL, USA); CLIA ACCESS SARS-CoV-2 IgG assay for the detection of the IgG antibodies against the receptor binding domain (RBD) in the S1 subunit of the spike protein (IgG anti-S1-RBD) (Beckman Coulter, Brea, CA, USA); CLIA LIAISON SARS-CoV-2 S2/S2 IgG assay for the detection of the IgG antibodies against the S1 and S2 subunit of the spike protein (IgG anti-S1/S2) (Diasorin, Saluggia, Italy); and an Anti-SARS-CoV-2 ELISA (IgG) assay for the detection of the IgG antibodies against the S1 subunit of the spike protein (IgG anti-S1) (Euroimmun, Lübeck, Germany). All tested assays calculated their results based on the measured signal production in the samples by dividing the stored calibrator result. The overview of the assays used is shown in Table 2.

2.4. Virus Neutralization Test (VNT)

VNTs were performed by the Department of Virology, Public Health Institute in Ostrava, Czech Republic, which is accredited for the performance of this method.
The VNT was performed using sterile 96-well plates. The SARS-CoV-2 strain was extracted from a clinical sample (hCoV-19/Czech Republic/NRL_9640/2020|EPI_ISL_626593) and CV-1 cells (African green monkey kidney fibroblasts) were used as the susceptible cell line; MEM (minimal essential medium) was buffered with 0.02 mmol/L TRIS-HCl pH 8.1 with the addition of 7.5% BFS (bovine fetal serum) and used as universal diluent. Serum samples were diluted in two-fold dilutions which, after mixing with the virus, resulted in a final serum concentration of 1/10, 1/20 … up to 1/2560. These dilutions were mixed with the virus solution (100 infection doses) and incubated overnight at +4 °C. The next day, 25 µL of CV-1 cell suspension was added into each well. The plates were incubated for further 3–4 days at 37 °C and 5% CO2 atmosphere. Then, 25 µL of neutral red dye (1:10,000 aqueous solution) was added into each well and the mixture was incubated for another day at 37 °C and 5% CO2. Then, the whole content of the wells was removed, and the uninfected results were red.
Only live uninfected cells were stained with the neutral red dye, enabling a macroscopic reading, and deeming the use of a microscope unnecessary. The results of the VNT were expressed in the form of virus neutralization titer, which represents an inverted value of the highest dilution of the sample neutralizing the cytopathic effect of the virus for more than 50%. The titer was calculated from the volume of serum and virus at the endpoint of the median tissue culture infectious dose (TCID50). Positivity was determined by a titer of 20 and above [13,14].
We compared individual immunoassays to each other. Subsequently, we compared the results of individual immunoassays with VNT. We created a visualization of the comparison of measured concentrations of SARS-CoV-2 antibodies by individual immunoassays, as well as an overview of the data in graphic form. We also assessed the dependence of neutralizing antibody production on age. After correlation with VNT, we suggested two cut-off levels for each immunoassay corresponding to the titers 80 and 160, respectively. We calculated sensitivity (the ability to correctly detect individuals with antibodies) and specificity (the ability to correctly detect individuals without antibodies).

3. Statistical Methods

SAS, V. 9.4 (SAS Institute Inc., Cary, NC, USA) and MATLAB, V. R2007b (The MathWorks, Inc., Natick, MA, USA) were used for all statistical analyses. Discrete characteristics are expressed as frequency counts and percentages; continuous characteristics are expressed as means and standard deviations or medians and quartiles, where appropriate.
The Spearman’s rank correlation was used to identify the best correlating immunoassays. The predictive power of the immunoassays was quantified by the area under the curve (AUC); receiver operating characteristic (ROC) curves were plotted, and cut-off levels, sensitivities and specificities were calculated. The cut-off level was defined as the 95% percentile of the group below a given titer (80 and 160, respectively). These 95% percentiles were calculated by the SAS proc univariate default method and the observation numbered closest to the number of non-missing results multiplied by p (where p = 0.95 in our case). ROC curves are presented for three of the best correlate assays.
Graphical statistical displays, histograms, and spaghetti plots are presented. The distribution of the results of a typical immunoassay was highly skewed towards high values; therefore, a linear (Pearson) correlation might be misleading, thus a Spearman’s rank correlation was used. Due to a relatively low number of patients, we advocated for the use of data visualization methods. We used the parallel coordinates plots to visualize the mutual agreement of the five immunoassays and their correspondence with the VNT results. Age dependence of measured parameters was also analyzed. This relationship can also be non-linear. Therefore, a scatter plot graph was first displayed, and then correlation by the Spearman’s correlation coefficient was calculated if the relationship to age was monotonous.

4. Results

The summary of the results of individual immunoassays used in the study is shown in Table 3. The values of medians depend on the design of individual assays. The highest median was achieved by Roche, following by Diasorin. The distribution of the results in individual assays determines the distance of the points from the x-axis in the visualization of the comparison of measured concentrations of SARS-CoV-2 antibodies (Figure 1).
The correlation between all immunoassays used in the study and the correlation of individual immunoassays with VNT are shown in Table 4. Assays targeted against spike protein (Diasorin, Beckman Coulter and Euroimmun) showed a correlation with each other. The correlation between the other two assays (Abbott and Roche) was weaker due to the different construction of the assays (detection of antibodies against the N protein). The correlation of individual immunoassays with the VNT reveals that assays targeted against the spike protein (Diasorin, Beckman Coulter and Euroimmun) showed correlation with the VNT, and assays targeted against the nucleocapsid protein (Abbott and Roche) exhibited less correlation. The closest correlation was demonstrated between VNT and the Diasorin assay.
Figure 1 represents the visualization of the comparison of measured concentrations of SARS-CoV-2 antibodies by individual assays and provides an overview of the data. The x-axis shows the individual methods for the determination of antibodies (Abbott, Diasorin, Roche, Beckman, Euroimmun). The y-axis plots the decimal logarithms of the standardized antibody values. Each test has a cut-off value below which the result is considered negative (1.4 for Abott, 12 for Diasorin, 1.0 for Roche, 0.8 for Beckman, and 0.8 for Euroimmun; see Table 1). Each measured value was divided by this threshold and then logarithmized. For example, the result of Diasorin test, 120, is thus transformed into log(120/12) = 1. This transformation ensures the comparability of individual methods with each other on the one hand (we convert everything to a multiple of the negativity limit) and, on the other, allows for a more even distribution of data points on the y-axis. Each yellow line in the graph connects values measured from the same serum sample by different methods. In the individual panels, only samples with the appropriate VNT antibody titer are marked in blue. In subgroup ≤40, all samples with the lowest titers 20 and 40 are marked in blue together; in subgroups 80, 160 and 320, the titers 80, 160 and 320 are plotted in blue; in subgroup ≥640, all samples where the highest titers are 640 and 1280 are marked in blue together. Samples with low titers of neutralizing antibodies are located at the bottom of the yellow “field”; a low titer of neutralizing antibodies corresponds to a low value of antibodies in all methods used. With increasing the VNT titer, blue lines appear at higher levels, until the highest titers are used at the top of the yellow field. In the case of VNT ≤ 40, one expects the blue lines to lie at the bottom of the yellow region. This happens very consistently in cases of Diasorin and Euroimmun, but it does not happen so consistently in the case of Roche.

4.1. Dependence of Neutralizing Antibody Titers on Age

The values of neutralizing antibody titers in the VNT test correlate significantly with the age of patients. The correlation between age and the result of the VNT test is low (c = 0.17), but marginally significant (p = 0.04). Figure 2 shows that elderly patients have a higher proportion of people with high VNT titers than young people. The percentage of higher titers (≥320) increases continuously with age, as seen in Figure 2b.

4.2. Cut-Off Values for Screening Suitable Donors of Convalescent Plasma

For the purpose of screening suitable convalescent plasma donors, we calculated cut-off values for individual immunochemical assays (Table 5). Based on these cut-off values of IgG antibodies measured in patients, it is possible to predict the values of neutralizing antibodies. The IgG antibody values measured by the Diasorin, Beckman Coulter, and Euroimmun assays correlated best with the VNT result (Table 4). Although the cut-off was defined as the 95% percentile of the group below a given titer, for some tests this led to 100% specificity at titer 80. The ROC curves for titer 80 have an atypical course and the 100% specificity was achieved randomly, which can be explained from the ROC curves (Figure 3a–c and Figure 4a–c). At a false positive of 0%, and a specificity of 100%, the curves increased for many cut-offs. For Roche and Abbot assays, their low correlation with the VTN caused lower sensitivities, especially at 160.

5. Discussion

Serological tests represent an important tool in our fight against the COVID-19 pandemic. Serological assays were very quickly developed, and in a short time their use gave rise to routine diagnostics capable of determining a patient’s previous contact or infection with the SARS-CoV-2 virus [15]. When we started routinely measuring SARS-CoV-2 antibodies in July 2020, we were presented with the opportunity to test and become acquainted with the five different immunoassays. Based on their biological differences, we can recognize two groups of immunoassays [16]: assays targeted against the spike protein, and assays targeted against the nucleocapsid protein. During the first wave of the pandemic, we primarily wanted to find out who had been in contact with the virus. For this purpose, the Roche assay seemed to be the best choice because of its construct. The concept of the total antibody measurement meant the highest sensitivity, which is visible in Figure 1 when the group with titer ≤40 was evaluated. The blue line which connects the values of the same samples in all the assays is most visible at the background of all (yellow) samples in the Roche assay. This finding is in concordance with the findings of other authors [17,18,19].
Experience with other strains of coronaviruses shows that the spike protein is the target for neutralizing antibodies. The nucleocapsid protein is shielded by a viral membrane, and antibodies targeted against the nucleocapsid protein cannot directly neutralize the virus [20]. Neutralizing antibodies protect the host by binding to specific proteins on the SARS-CoV-2 virus to neutralize its ability to bind to the host cellular receptor angiotensin-converting enzyme 2 (ACE2), and after the fusion of viral and cellular membranes, enter into the host cell. Antibodies also facilitate the recognition and killing of the virus by phagocytic immune cells [21].
It was on the basis of this that COVID-19 convalescent plasma administration started in many hospitals as the experimental treatment option; as passive antibody therapy [11]. According to the current recommendation, the convalescent plasma should have at least a 160 titer of the neutralizing antibodies, although titer 80 is still acceptable as a minimum threshold value [22]. To assess the ability of tested assays to predict the presence of neutralizing antibodies, we performed VNT and compared the results of titers with the values of SARS-CoV-2 antibodies measured by individual assays. In agreement with other studies [23,24,25,26], we have proved a very good correlation of anti-spike protein assays with the VNT. The blue lines in anti-spike protein assays are visible in Figure 1 (Diasorin, Beckman Coulter and Euroimmun) in individual subgroups (starting with the titer ≤40 to the final subgroup titer ≥640); they pass more evenly through the diagram from the bottom to the top, which is a sign of the individual assays correlating better with the VNT. Correlation coefficients are presented in Table 3. To assure the titer of the neutralizing antibodies at 80 or 160, respectively, we suggested a cut-off level and calculated sensitivity and specificity. We also plotted ROC curves and calculated AUC values for each assay (Table 4). It can be seen from Table 4 that the assays targeted against the nucleocapsid protein mainly provide information about the contact with the virus. In our evaluation, these assays achieved very low sensitivity, especially for titer 160. Therefore, ROC curves are presented only for anti-spike protein assays (Figure 3a–c and Figure 4a–c), potential predictors of the values of neutralizing antibodies. Cut-off levels for Euroimmun and Diasorin can be compared with the available study by Valdivia et al. [26].
Valdivia et al. worked with a group of 51 patients, which is a smaller group than ours, and the patients were hospitalized, which means that the course of the diseases was probably worse than in our group of patients. However, at the current level of knowledge, it is hard to hypothesize the affect that this difference could have had on the results of the serology tests obtained in both studies. They targeted a neutralization titer of 160 and 95% specificity. They achieved higher cut-off levels for Euroimmun (8.9) compared to our cut-off value (3.06). However, for Diasorin assay, the cut-offs were closer together: the cut-off by Valdivia et al. was 90.6 AU/mL, compared to our cut-off 73.2 AU/mL.
Based on these findings, we can conclude that the cut-off values of anti-spike protein assays shown in Table 4 can be added to the general rules for searching for convalescent plasma donors (patients with the confirmed COVID-19 infection who have recovered). Adding the cut-off values of anti-spike protein assays from Table 4 as additional criteria to the screening of the suitable donors of the convalescent plasma will ensure a high probability of titers of neutralizing antibodies.
The last issue we wanted to address is the correlation of the production of neutralizing antibodies with age. Such information is still rare, and some authors point out the necessity of obtaining such data [27]. We tried to create a homogeneous group of patients with respect to the course of the disease, so that the levels of antibodies were comparable across the age spectrum. The patients enrolled in the study were not hospitalized, all with mild clinical symptoms. The blood samples were taken at the same time period from the onset of the disease. Spearman’s rank correlation was used, which cannot be influenced by outliers. In accordance with another Czech study [13], we proved a positive correlation of the titer of neutralizing antibodies with the increasing age of patients (Figure 2).

6. Conclusions

Serological assays that measure SARS-CoV-2 antibodies are helpful tools in the COVID-19 pandemic situation. The assays can provide information about the patient’s contact with the virus. Anti-spike protein assays correlate well with the VNT and can be used to screen potential convalescent plasma donors. In our group of patients, we proved a positive correlation between the titer of neutralizing antibodies and older age groups.

Author Contributions

Conceptualization, V.Š.; methodology, L.P., O.T., H.Z.; formal analysis, L.P., T.F., Z.K.; investigation, K.F., D.S. (Dalibor Sedláček), R.Š., P.P.; data curation, R.K.; writing—original draft preparation, R.K., Z.K., L.P., T.F.; writing—review and editing, V.Š., K.F., D.S. (David Slouka); H.Ř.; visualization, L.P., T.F.; supervision, V.Š. All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported by a grant of the Ministry of Health of the Czech Republic—Conceptual Development of Research Organization (University Hospital Pilsen—FNPl, 00669806); BBMRI-CZ: Biobank network—a versatile platform for the research of the etiopathogenesis of diseases (CZ.02.1.01/0.0/0.0/16_013/0001674); and Bank of the clinical samples (LM2018125).

Institutional Review Board Statement

The study was conducted according to the guidelines of the Declaration of Helsinki, and approved by the Ethics Committee of University Hospital and Medical Faculty in Pilsen on 8 August 2020 (approval number 354/2020).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

Data are available by the corresponding author.

Conflicts of Interest

The authors declare no competing interests.

References

  1. Huang, C.; Wang, Y.; Li, X.; Ren, L.; Zhao, J.; Hu, Y.; Zhang, L.; Fan, G.; Xu, J.; Gu, X.; et al. Clinical Features of Patients Infected with 2019 Novel Coronavirus in Wuhan, China. Lancet 2020, 395, 497–506. [Google Scholar] [CrossRef][Green Version]
  2. COVID-19 Dashboard by the Center for Systems Science and Engineering (CSSE) at Johns Hopkins University (JHU). Available online: https://www.arcgis.com/apps/opsdashboard/index.html#/bda7594740fd40299423467b48e9ecf6 (accessed on 2 March 2021).
  3. Poland, G.A.; Ovsyannikova, I.G.; Kennedy, R.B. SARS-CoV-2 Immunity: Review and Applications to Phase 3 Vaccine Candidates. Lancet 2020, 396, 1595–1606. [Google Scholar] [CrossRef]
  4. Sekine, T.; Perez-Potti, A.; Rivera-Ballesteros, O.; Strålin, K.; Gorin, J.-B.; Olsson, A.; Llewellyn-Lacey, S.; Kamal, H.; Bogdanovic, G.; Muschiol, S.; et al. Robust T Cell Immunity in Convalescent Individuals with Asymptomatic or Mild COVID-19. Cell 2020, 183, 158–168.e14. [Google Scholar] [CrossRef]
  5. Kellam, P.; Barclay, W. The Dynamics of Humoral Immune Responses Following SARS-CoV-2 Infection and the Potential for Reinfection. J. Gen. Virol. 2020, 101, 791–797. [Google Scholar] [CrossRef]
  6. Callow, K.A.; Parry, H.F.; Sergeant, M.; Tyrrell, D.A. The Time Course of the Immune Response to Experimental Coronavirus Infection of Man. Epidemiol. Infect. 1990, 105, 435–446. [Google Scholar] [CrossRef] [PubMed][Green Version]
  7. Ledford, H. Coronavirus Reinfections: Three Questions Scientists Are Asking. Nature 2020, 585, 168–169. [Google Scholar] [CrossRef] [PubMed]
  8. Pickering, S.; Betancor, G.; Galão, R.P.; Merrick, B.; Signell, A.W.; Wilson, H.D.; Kia Ik, M.T.; Seow, J.; Graham, C.; Acors, S.; et al. Comparative Assessment of Multiple COVID-19 Serological Technologies Supports Continued Evaluation of Point-of-Care Lateral Flow Assays in Hospital and Community Healthcare Settings. PLoS Pathog. 2020, 16, e1008817. [Google Scholar] [CrossRef]
  9. Wrapp, D.; Wang, N.; Corbett, K.S.; Goldsmith, J.A.; Hsieh, C.-L.; Abiona, O.; Graham, B.S.; McLellan, J.S. Cryo-EM Structure of the 2019-NCoV Spike in the Prefusion Conformation. Science 2020, 367, 1260–1263. [Google Scholar] [CrossRef][Green Version]
  10. Litzba, N.; Zelená, H.; Kreil, T.R.; Niklasson, B.; Kühlmann-Rabens, I.; Remoli, M.E.; Niedrig, M. Evaluation of Different Serological Diagnostic Methods for Tick-Borne Encephalitis Virus: Enzyme-Linked Immunosorbent, Immunofluorescence, and Neutralization Assay. Vector Borne Zoonotic Dis. 2014, 14, 149–159. [Google Scholar] [CrossRef] [PubMed]
  11. Casadevall, A.; Pirofski, L. The Convalescent Sera Option for Containing COVID-19. J. Clin. Investig. 2020, 130, 1545–1548. [Google Scholar] [CrossRef] [PubMed][Green Version]
  12. Duan, K.; Liu, B.; Li, C.; Zhang, H.; Yu, T.; Qu, J.; Zhou, M.; Chen, L.; Meng, S.; Hu, Y.; et al. Effectiveness of Convalescent Plasma Therapy in Severe COVID-19 Patients. Proc. Natl. Acad. Sci. USA 2020, 117, 9490–9496. [Google Scholar] [CrossRef] [PubMed][Green Version]
  13. Fejt, V.; Krátká, Z.; Zelená, H.; Fürst, T. Age Is Not a Disease: Evolution of Protective Antibodies against SARS-CoV-2 in Seniors from the Brevnice Nursing Home. Cas Lek Cesk 2020, 159, 303–311. [Google Scholar] [PubMed]
  14. Valcourt, E.J.; Manguiat, K.; Robinson, A.; Chen, J.C.-Y.; Dimitrova, K.; Philipson, C.; Lamoureux, L.; McLachlan, E.; Schiffman, Z.; Drebot, M.A.; et al. Evaluation of a Commercially-Available Surrogate Virus Neutralization Test for Severe Acute Respiratory Syndrome Coronavirus-2 (SARS-CoV-2). Diagn. Microbiol. Infect. Dis. 2021, 99, 115294. [Google Scholar] [CrossRef]
  15. Yongchen, Z.; Shen, H.; Wang, X.; Shi, X.; Li, Y.; Yan, J.; Chen, Y.; Gu, B. Different Longitudinal Patterns of Nucleic Acid and Serology Testing Results Based on Disease Severity of COVID-19 Patients. Emerg. Microbes Infect. 2020, 9, 833–836. [Google Scholar] [CrossRef][Green Version]
  16. Letko, M.; Marzi, A.; Munster, V. Functional Assessment of Cell Entry and Receptor Usage for SARS-CoV-2 and Other Lineage B Betacoronaviruses. Nat. Microbiol. 2020, 5, 562–569. [Google Scholar] [CrossRef][Green Version]
  17. Afzal, N.; Tariq, N.; Raza, S.; Shakeel, D. Diagnostic Accuracy of Electro-Chemiluminescence Immunoassay Anti-SARS-CoV-2 Serological Test. Cureus 2021, 13, e12588. [Google Scholar]
  18. Irsara, C.; Egger, A.E.; Prokop, W.; Nairz, M.; Loacker, L.; Sahanic, S.; Pizzini, A.; Sonnweber, T.; Mayer, W.; Schennach, H.; et al. Evaluation of Four Commercial, Fully Automated SARS-CoV-2 Antibody Tests Suggests a Revision of the Siemens SARS-CoV-2 IgG Assay. Clin. Chem. Lab. Med. 2021. published Ahead of print. [Google Scholar] [CrossRef]
  19. Weidner, L.; Gänsdorfer, S.; Unterweger, S.; Weseslindtner, L.; Drexler, C.; Farcet, M.; Witt, V.; Schistal, E.; Schlenke, P.; Kreil, T.R.; et al. Quantification of SARS-CoV-2 Antibodies with Eight Commercially Available Immunoassays. J. Clin. Virol. 2020, 129, 104540. [Google Scholar] [CrossRef]
  20. Ghaffari, A.; Meurant, R.; Ardakani, A. COVID-19 Serological Tests: How Well Do They Actually Perform? Diagnostics 2020, 10, 453. [Google Scholar] [CrossRef]
  21. Amanat, F.; Krammer, F. SARS-CoV-2 Vaccines: Status Report. Immunity 2020, 52, 583–589. [Google Scholar] [CrossRef]
  22. Freedenberg, A.T.; Pan, C.; Diehl, W.E.; Romeiser, J.L.; Hwang, G.; Leiton, C.V.; Muecksch, F.; Shroyer, K.R.; Bennett-Guerrero, E. Neutralizing Activity to SARS-CoV-2 of Convalescent and Control Plasma Used in a Randomized Controlled Trial. Transfusion 2021, 1–7. [Google Scholar] [CrossRef]
  23. Salazar, E.; Kuchipudi, S.V.; Christensen, P.A.; Eagar, T.N.; Yi, X.; Zhao, P.; Jin, Z.; Long, S.W.; Olsen, R.J.; Chen, J.; et al. Relationship between Anti-Spike Protein Antibody Titers and SARS-CoV-2 In Vitro Virus Neutralization in Convalescent Plasma. bioRxiv 2020. [Google Scholar] [CrossRef]
  24. Suhandynata, R.T.; Hoffman, M.A.; Huang, D.; Tran, J.T.; Kelner, M.J.; Reed, S.L.; McLawhon, R.W.; Voss, J.E.; Nemazee, D.; Fitzgerald, R.L. Commercial Serology Assays Predict Neutralization Activity against SARS-CoV-2. Clin. Chem. 2021, 67, 404–414. [Google Scholar] [CrossRef]
  25. Gniadek, T.J.; Thiede, J.M.; Matchett, W.E.; Gress, A.R.; Pape, K.A.; Fiege, J.K.; Jenkins, M.K.; Menachery, V.D.; Langlois, R.A.; Bold, T.D. SARS-CoV-2 Neutralization and Serology Testing of COVID-19 Convalescent Plasma from Donors with Nonsevere Disease. bioRxiv 2020. [Google Scholar] [CrossRef] [PubMed]
  26. Valdivia, A.; Torres, I.; Latorre, V.; Francés-Gómez, C.; Albert, E.; Gozalbo-Rovira, R.; Alcaraz, M.J.; Buesa, J.; Rodríguez-Díaz, J.; Geller, R.; et al. Inference of SARS-CoV-2 Spike-Binding Neutralizing Antibody Titers in Sera from Hospitalized COVID-19 Patients by Using Commercial Enzyme and Chemiluminescent Immunoassays. Eur. J. Clin. Microbiol. Infect. Dis. 2021, 40, 485–494. [Google Scholar] [CrossRef] [PubMed]
  27. Post, N.; Eddy, D.; Huntley, C.; van Schalkwyk, M.C.I.; Shrotri, M.; Leeman, D.; Rigby, S.; Williams, S.V.; Bermingham, W.H.; Kellam, P.; et al. Antibody Response to SARS-CoV-2 Infection in Humans: A Systematic Review. PLoS ONE 2020, 15, e0244126. [Google Scholar] [CrossRef]
Figure 1. Visualization of the comparison of measured concentrations of SARS-CoV-2 antibodies by individual assays and overview of the data. Legends: The results of each test were standardized by taking the (decadic) logarithm of the ratio of the test result to the cut-off for negativity (see Table 4 and the text for more details). Each yellow line represents a single sample. In each panel, samples of particular VNT value are highlighted blue. We may observe the overall agreement of the five tests (Table 3) and the general correspondence of the VNT results with the five tests.
Figure 1. Visualization of the comparison of measured concentrations of SARS-CoV-2 antibodies by individual assays and overview of the data. Legends: The results of each test were standardized by taking the (decadic) logarithm of the ratio of the test result to the cut-off for negativity (see Table 4 and the text for more details). Each yellow line represents a single sample. In each panel, samples of particular VNT value are highlighted blue. We may observe the overall agreement of the five tests (Table 3) and the general correspondence of the VNT results with the five tests.
Diagnostics 11 00593 g001
Figure 2. (a) The number of VNT results across several age groups. (b) The proportion of VNT results across several age groups. Legend: The proportion of VNT ≥ 320 subjects visibly grows with age, while the proportion of VNT ≤ 80 decreases.
Figure 2. (a) The number of VNT results across several age groups. (b) The proportion of VNT results across several age groups. Legend: The proportion of VNT ≥ 320 subjects visibly grows with age, while the proportion of VNT ≤ 80 decreases.
Diagnostics 11 00593 g002
Figure 3. (a) Diasorin cut-off for titer 80. (b) Beckman Coulter cut-off for titer 80. (c) Euroimmun cut-off for titer 80. Legend: ROC curve for Diasorin, Beckman Coulter and Euroimmun, titer <80 vs. ≥80 where the ROC curve is labeled by all possible cut-offs. These ROC curves are atypical, and, above all, their vertical parts increase at the point of 100% specificity, demonstrating that it was not targeted at 100% specificity, but 95%. For some methodologies, however, it accidentally led to 100% specificity. For cut-off values for individual methods, see Table 4.
Figure 3. (a) Diasorin cut-off for titer 80. (b) Beckman Coulter cut-off for titer 80. (c) Euroimmun cut-off for titer 80. Legend: ROC curve for Diasorin, Beckman Coulter and Euroimmun, titer <80 vs. ≥80 where the ROC curve is labeled by all possible cut-offs. These ROC curves are atypical, and, above all, their vertical parts increase at the point of 100% specificity, demonstrating that it was not targeted at 100% specificity, but 95%. For some methodologies, however, it accidentally led to 100% specificity. For cut-off values for individual methods, see Table 4.
Diagnostics 11 00593 g003aDiagnostics 11 00593 g003b
Figure 4. (a) Diasorin cut-off for titer 160. (b) Beckman Coulter cut-off for titer 160. (c) Euroimmun cut-off for titer 160. Legend: ROC curve for Diasorin, Beckman Coulter and Euroimmun, titer <160 vs. ≥160 where the ROC curve is labeled by all possible cut-offs. These ROC curves are already typical. Thus, specificity can be targeted as close to 95%. For cut-off values for individual methods, see Table 4.
Figure 4. (a) Diasorin cut-off for titer 160. (b) Beckman Coulter cut-off for titer 160. (c) Euroimmun cut-off for titer 160. Legend: ROC curve for Diasorin, Beckman Coulter and Euroimmun, titer <160 vs. ≥160 where the ROC curve is labeled by all possible cut-offs. These ROC curves are already typical. Thus, specificity can be targeted as close to 95%. For cut-off values for individual methods, see Table 4.
Diagnostics 11 00593 g004aDiagnostics 11 00593 g004b
Table 1. Clinical characteristics of the patient group.
Table 1. Clinical characteristics of the patient group.
CharacteristicCategoryn (%)
GenderFemale94 (63)
Male55 (37)
Total149 (100)
AgeMedian [range]47.5 (4–73)
PCRpositivity/negativity138 (92)/11 (8)
Clinical symptomsfever ≥ 37.5 °C69 (46)
dry cough51 (34)
difficulty breathing or shortness of breath33 (22)
loss of taste or smell59 (39)
headache68 (45)
tiredness105 (70)
Risk factorsobesity17 (11)
smoking (last 10 years)17 (11)
diabetes12 (8)
cardiovascular disease13 (9)
Flu vaccinationfor season 20199 (6)
Table 2. Immunoassays of SARS-CoV-2 antibodies used in the study.
Table 2. Immunoassays of SARS-CoV-2 antibodies used in the study.
ManufacturerMethodologyAntigen UsedEvaluation of the ResultsManufacturer’s
Catalog Number
NegativeGray ZonePositive
AbbottCLIAIgG NP<1.4-≥1.46R86
DiasorinCLIAIgG anti-S1/S2<1212–15≥15311450
RocheCLIATotal Ig NP<1.0-≥1.009 203 095 190
Beckman CoulterCLIAIgG anti-S1 RBD<0.80.8–1.0≥1.0C58961
EuroimmunELISAIgG anti-S1<0.80.8–1.1≥1.1EI 2606-9601 G
Legend: The individual assays report the results as follows: Abbott as an index value (S/C); Diasorin as arbitrary units per mililiter (AU/mL); Roche as cut-off index (COI); Beckman Coulter as an index value (S/CO); Euroimmun as a ratio (number only).
Table 3. Summary of the results of individual immunoassays used.
Table 3. Summary of the results of individual immunoassays used.
ManufacturernMedianMinimumMaximumLower QuartileUpper Quartile
Abbott1492.660.1010.121.514.91
Diasorin14966.505.5440034.5110
Roche14973.451.8415531.8107
Beckman Coulter1491.880.1240.20.725.47
Euroimmun1492.460.128.612.464.54
Table 4. Correlation of the individual immunoassays for the determination of SARS-CoV-2 antibodies and their correlation with the virus neutralization test (VNT).
Table 4. Correlation of the individual immunoassays for the determination of SARS-CoV-2 antibodies and their correlation with the virus neutralization test (VNT).
AbbottDiasorinRocheBeckmanEuroimmunVNT
Abott1.000.490.650.650.490.49
Diasorin 1.000.410.760.840.72
Roche 1.000.390.380.38
Beckman Coulter 1.000.730.68
Euroimmun 1.000.63
VNT 1.00
Legends: All the (Spearman’s) correlation coefficients are significant at p < 0.001. The diagonal is highlighted in yellow, correlations above 0.7 are marked in red, and correlations below 0.5 are marked in blue.
Table 5. Cut-off value with sensitivity and specificity for VNT titer 80 and 160.
Table 5. Cut-off value with sensitivity and specificity for VNT titer 80 and 160.
ManufacturerTiter 80Titer 160
Cut-OffSensitivitySpecificityAUCCut-OffSensitivitySpecificityAUC
Diasorin35.182%100%0.934173.264%96%0.8636
Beckman Coulter2.3949%100%0.87773.3150%96%0.8435
Euroimmun2.0965%100%0.88063.0655%96%0.8258
Abbott4.2635%100%0.88196.0026%96%0.7168
Roche68.459%94%0.8514131.413%96%0.7111
Legend: The individual assays report the results as follows: Abbott as an index value (S/C); Diasorin as arbitrary units per milliliter (AU/mL); Roche as cut-off index (COI); Beckman Coulter as an index value (S/CO); Euroimmun as a ratio (number only); Cut-off is defined as the 95% percentile of the group below the titer 80 and 160; AUC, area under the curve.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Šimánek, V.; Pecen, L.; Krátká, Z.; Fürst, T.; Řezáčková, H.; Topolčan, O.; Fajfrlík, K.; Sedláček, D.; Šín, R.; Pazdiora, P.; Zelená, H.; Slouka, D.; Kučera, R. Five Commercial Immunoassays for SARS-CoV-2 Antibody Determination and Their Comparison and Correlation with the Virus Neutralization Test. Diagnostics 2021, 11, 593. https://doi.org/10.3390/diagnostics11040593

AMA Style

Šimánek V, Pecen L, Krátká Z, Fürst T, Řezáčková H, Topolčan O, Fajfrlík K, Sedláček D, Šín R, Pazdiora P, Zelená H, Slouka D, Kučera R. Five Commercial Immunoassays for SARS-CoV-2 Antibody Determination and Their Comparison and Correlation with the Virus Neutralization Test. Diagnostics. 2021; 11(4):593. https://doi.org/10.3390/diagnostics11040593

Chicago/Turabian Style

Šimánek, Václav, Ladislav Pecen, Zuzana Krátká, Tomáš Fürst, Hana Řezáčková, Ondřej Topolčan, Karel Fajfrlík, Dalibor Sedláček, Robin Šín, Petr Pazdiora, Hana Zelená, David Slouka, and Radek Kučera. 2021. "Five Commercial Immunoassays for SARS-CoV-2 Antibody Determination and Their Comparison and Correlation with the Virus Neutralization Test" Diagnostics 11, no. 4: 593. https://doi.org/10.3390/diagnostics11040593

Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop