Performance Evaluation of Three Antibody Binding Assays, a Neutralizing Antibody Assay, and an Interferon-Gamma Release Assay for SARS-CoV-2 According to Vaccine Type in Vaccinated Group

We evaluated the performance of SARS-CoV-2 assays in the vaccinated group using receptor-binding domain antibody assays (RBD Ab assay), neutralizing antibody assay (nAb assay), and interferon-gamma release assay (IGR assay). We also compared the performance of the SARS-CoV-2 assays based on vaccine type in a large population. We collected 1851 samples from vaccinated individuals with vector, mix-and-match (MM), and mRNA vaccines. The performance of the RBD Ab assays was assessed by SARS-CoV-2 IgG II Quant (Abbott Laboratories, Sligo, Ireland), SARS-CoV-2 IgG (Beckman Coulter, CA, USA), and anti-SARS-CoV-2 S (Roche Diagnostics GmbH, Mannheim, Germany). The nAb assay was assessed by cPass SARS-CoV-2 neutralization antibody detection kits (GenScript, NJ, USA). The IGR assay was assessed by QuantiFERON (Qiagen, Venlo, The Netherlands). Median values of the RBD Ab assays and nAb assay sequentially increased after the first and second vaccinations. RBD Ab assays and nAb assay showed very strong correlations. The median values of the RBD Ab, nAb, and IGR were higher in the mRNA vaccine group than in the vector and MM vaccine groups. The agreement and correlation among the RBD Ab assays, nAb assay, and IGR assay were higher in the mRNA vaccine group than in the vector and MM vaccine groups. We compared the performance of the RBD Ab assay, nAb assay, and IGR assay based on the vaccine types using the RBD Ab, nAb, and IGR assays. This study provides a better understanding of the assessment of humoral and cellular immune responses after vaccination.


Introduction
Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has spread globally, causing widespread mortality and morbidity [1].Immunization emerges as the preeminent and secure strategy for mitigating the impact of the ongoing pandemic, demonstrating unparalleled safety and efficacy in curtailing the spread of the infectious agent and averting severe clinical outcomes.The population acquires humoral or cellular immune responses against SARS-CoV-2 through vaccination and natural infection [2][3][4].The immune response can be assessed by several tests including the SARS-CoV-2 antibody binding assay (Ab assay), SARS-CoV-2 neutralizing antibody assay (nAb assay), and interferon-gamma release assay (IGR assay) [3,4].
The Ab assay measures the humoral immune responses after vaccination and natural infection and has been developed for different target antigens and assay platforms [5].This Ab assay mainly detects antibodies (Ab) (IgM, IgG, IgA, or total Ab) against the receptor-binding domain (RBD) of the spike protein (S), partial S protein, or nucleocapsid (N) protein [6].Many Ab assays with high sensitivity and specificity can be harmonized and validated using World Health Organization (WHO) International Standard (IS) [7].The nAb assay also measures the humoral immune responses that defend against blocking viral entry into host cells and neutralizing their biological effect in collaboration with immune cells [8].
A commercialized nAb assay assesses potential values as a surrogate for nAb that blocks the interactions between RBD and the angiotensin-converting enzyme 2 (ACE2) receptor [9].The SARS-CoV-2 surrogate virus neutralizing test (sVNT) (GenScript Inc., Leiden, The Netherlands), obtained the first emergency use authorization for nAb detection.It is easily performed in clinical laboratories within a few hours using a commercial assay [10].
Many commercial assays measuring humoral immune responses are available.However, assessing the cellular immune responses is essential for understanding long-term immunity, vaccine effectiveness, and vaccine durability as memory T cells play a key role in providing sustained protection beyond the humoral immune responses [11].IGR assays on the enzyme-linked immunosorbent assay (ELISA) platform are commonly used to detect the cellular immune responses after cytomegalovirus or Mycobacterium tuberculosis infection by measuring IGR values [12,13].The IGR assay could be applied to SARS-CoV-2 infection and was designed to function without special equipment in a short testing time [14].
Many studies have evaluated the performance of RBD antibody (Ab) assays and the nAb assay among vaccinated groups.Depending on the RBD Ab assay platform, RBD Ab assays had a sensitivity of 90.0% to 97.4% and a specificity of 97.9% to 100% [15].Moreover, the RBD Ab assay and nAb assay were associated to varying degrees [16][17][18][19].Unlike commercial assays measuring humoral immune responses, the conventional methods of detecting cellular immune responses are complex and have not yet been standardized, requiring highly specialized facilities [20][21][22].Moreover, few studies have comprehensively compared the performance of RBD Ab assay, nAb assay, and IGR assay for SARS-CoV-2 in a group of vaccinated people.
RBD Ab assays, nAb assays, and IGR assays are commercially available to measure levels of humoral and cellular immunity.Therefore, we aimed to evaluate the analytical performance of the RBD Ab assay, nAb assay, and IGR assay based on the vaccine type, vector, mix-and-match (MM), and mRNA vaccines, in a large population.

Study Samples
This study was conducted from March 2021 to May 2022 at the Korea University Anam Hospital, Seoul, Republic of Korea.We included subjects aged 18 years and above, comprising vaccinated subjects without a history of infection.The subjects received the vector vaccine (ChAdOx1 nCoV-19, AstraZeneca, Cambridge, UK), MM vaccine (1st vaccination: ChAdOx1 nCoV-19; 2nd vaccination mRNA vaccine), or mRNA vaccine (BNT162b2, Pfizer-BioNTech, Pfizer Inc., New York, NY, USA; mRNA-1273, Moderna, Inc., Cambridge, MA, USA).Subjects were excluded if they: (1) were less than 18 years old, (2) had a history of SARS-CoV-2 infection, or (3) were on medication with immunosuppressants.All subjects were provided written informed consent before enrollment and a questionnaire at every blood collection.The questionnaire inquired about current COVID-19 symptoms, a history of prior infection, contact with confirmed patients, adverse responses following vaccination, and underlying disease.Enrolled subjects were tested for N Ab, RBD Ab, or nAb to monitor unknown natural infection.Despite having no history of natural infection, subjects with clinically suspect symptoms underwent reverse transcription-polymerase chain reaction to confirm SARS-CoV-2 infection.Subjects exhibiting positive results were subsequently excluded.The serum samples for each vaccinated subject were obtained three times: T0 for pre-vaccination, T1 for time points of 3-4 weeks (mRNA vaccine) or 7-8 weeks (vector vaccine) after their 1st vaccination, and T2 for the time point of 3-4 weeks after their 2nd vaccination.This study included 1851 serum samples obtained from 733 vaccinated subjects (vector, 484 subjects; MM, 145 subjects; mRNA 104 subjects).This study was approved by the Institutional Review Board of KUAH (K2021-0511-014).Informed consent was obtained from all vaccinated subjects before sample collection.

SARS-CoV-2 IgG Antibody Binding Assays
The vaccinated group was evaluated using Alinity SARS-CoV-2 IgG II Quant (Abbott Laboratories, Sligo, Ireland) (RBD Ab_A), Access SARS-CoV-2 IgG (BeckmanCoulter Inc., CA, USA) (RBD Ab_B), and Elecsys anti-SARS-CoV-2 S (Roche Diagnostics GmbH, Mannheim, Germany) (RBD Ab_R) for RBD Ab assays according to the manufacturer's instructions.RBD Ab_A, RBD Ab_B, and RBD Ab_R measured IgG Abs against the RBD of the spike protein in SARS-CoV-2.RBD Ab_A, RBD Ab_B, and RBD Ab_R were designed to provide qualitative and quantitative determination of SARS-CoV-2 IgG Abs.RBD Ab_A is an automated two-step sandwich immunoassay using indirect chemiluminescent microparticle technology.Antibody concentration is presented using arbitrary units (AU/mL).Arbitrary units were converted to binding antibody units per milliliter (BAU/mL), which are WHO international standards for anti-SARS-CoV-2 immunoglobulin, using the following equation: BAU/mL = 0.142 × AU/mL.The cutoff for the determination of a negative or positive value was 7.1 BAU/mL.The analytical measurement interval (AMR) is 3.0-5680 BAU/mL, and the extended measuring interval (EMI) by dilution was 5680-11,360 BAU/mL.RBD Ab_B is an automated two-step enzyme immunoassay using a magnetic particle-chemiluminescence enzyme technology.The antibody concentration was determined as a non-reactive or reactive response based on a cutoff of 30 BAU/mL.The AMR is 8.0-1800 BAU/mL and the EMI is 1500-36,000 BAU/mL.RBD Ab_R is an automated two-step sandwich immunoassay using electrochemiluminescence technology.The antibody concentration is presented using units of U/mL and was converted to BAU/mL using the following equation: BAU/mL = 0.972 × U/mL.The cutoff for antibody concentration was 0.8 BAU/mL.The AMR is 0.4-243.0BAU/mL, and results over 243 BAU/mL were diluted and re-tested to obtain accurate results.

SARS-CoV-2 Neutralizing Antibody Assay
The vaccinated group was evaluated using the GenScript for the nAb assay according to the manufacturer's instructions.The nAb values were measured by competitive ELISA.The ACE2 receptor precoated on the microplate was incubated with horseradish peroxidase(HRP)-labeled RBD, producing a strong signal.If neutralizing antibodies were present in the sample, they would attach to HRP-labeled RBD and protect it from binding to the microplate's ACE2 receptor.A low signal is produced by serum samples containing more nAb.The signal (optical density, OD) was measured at 450 nm on a spectrophotometer and was calculated using the following formula: signal inhibition (%) = (1 − OD value of sample/OD value of negative control) × 100.The signal inhibition results were converted to IU/mL according to the formula suggested by the manufacturer.The nAb values were determined to be a negative or positive response based on a cutoff of ≥28 IU/mL.

Interferon-Gamma Release Assay for SARS-CoV-2
The vaccinated group was evaluated using the QuantiFERON for the IGR assay (Qiagen, Venlo, The Netherlands, research use only) according to the manufacturer's instructions.Each 1 mL blood sample was directly placed into four QuantiFERON SARS-CoV-2 blood collection tubes (Qiagen): a nil tube, an Ag1 tube containing CD4 + epitopes of the S1 subunits (class II), an Ag2 tube containing CD4 + T cell and CD8 + T cell-specific epitopes (class I/II) of the S1 and S2 subunits, and the mitogen tube.The nil tube was used to calculate the background signal, and the mitogen tube was used as a positive control.IGR values were determined by subtracting the values for the nil tube from those for the class II or class I/II tube.The manufacturer did not suggest a definite cutoff.We determined the cutoff by a receiver operating characteristic curve where the cutoff of class II was >0.037 IU/mL and that of class I/II was >0.052 IU/mL.

Comparison of Qualitative and Quantitative Values in the Vaccinated Group
The median values of the RBD Ab, nAb, and IGR assays were sequentially increased after T1 and T2 (Table 1).The TA values were over 79.4% (range 79.4-98.4%)(Table 2).The agreement between RBD Ab_A and RBD Ab_R was almost perfect (kappa 0.96, 95% CI 0.95-0.98).Among three RBD Ab assays, RBD Ab_B showed the highest agreement with the nAb assay (kappa 0.83, 95% CI 0.80-0.85)and the lowest agreement with the IGR assay (kappa 0.55, 95% CI 0.51-0.60).Agreement between the nAb assay and the IGR assay was weak (kappa 0.58, 95% CI 0.54-0.63).The quantitative values among RBD Ab assays and nAb assays showed very strong associations (rho range 0.86-0.95) and those between the IGR assay and other assays showed strong to very strong associations (rho range 0.57-0.96)(Figure 1).

Comparison of Qualitative and Quantitative Values according to the Vaccine Types
The median values of the RBD Ab, nAb, and IGR assays were higher in the MM vaccine group or mRNA vaccine group than in the vector vaccine group (Table 1).The ranges of the TA values were 75.4%-98.3% in the vector vaccine group, 80.8%-98.2% in the MM vaccine group, and 89.4%-99.6% in the mRNA vaccine group (Table 3).The agreements were the highest in the mRNA vaccine group (kappa range 0.76-0.99)(Table 3).The agreements were relatively low between the IGR assay and other assays (kappa range 0.48-0.79).Regarding the quantitative values, the ranges of the rho correlation coefficient were 0.39-0.96 in the vector vaccine group, 0.62-0.98 in the MM vaccine group, and 0.78-0.96 in the mRNA vaccine group (Figure 2).The rho correlation coefficients between the RBD Ab assays and nAb assay were higher in the MM vaccine group and those between IGR and other assays were higher in the mRNA vaccine group (Table 4).

Comparison of Qualitative and Quantitative Values according to the Vaccine Types
The median values of the RBD Ab, nAb, and IGR assays were higher in the MM vaccine group or mRNA vaccine group than in the vector vaccine group (Table 1).The ranges of the TA values were 75.4%-98.3% in the vector vaccine group, 80.8%-98.2% in the MM vaccine group, and 89.4%-99.6% in the mRNA vaccine group (Table 3).The agreements were the highest in the mRNA vaccine group (kappa range 0.76-0.99)(Table 3).The agreements were relatively low between the IGR assay and other assays (kappa range 0.48-0.79).Regarding the quantitative values, the ranges of the rho correlation coefficient were 0.39-0.96 in the vector vaccine group, 0.62-0.98 in the MM vaccine group, and 0.78-0.96 in the mRNA vaccine group (Figure 2).The rho correlation coefficients between the RBD Ab assays and nAb assay were higher in the MM vaccine group and those between IGR and other assays were higher in the mRNA vaccine group (Table 4).Abbreviations: See Tables 1 and 2.

Discussion
There are currently numerous anti-SARS-CoV-2 antibody immunoassays on the market.This can be overwhelming for laboratory directors who need to choose which test to use in routine practice.At the start of the pandemic, these immunoassays were primarily used to confirm SARS-CoV-2 infection and there were debates about which tests showed better performance based on N or S (RBD) proteins [25,26].The rollout of vaccines in late 2020 resulted in a preference for RBD Ab assays in assessing the humoral immune response to the COVID-19 vaccine and its role in estimating protective immunity became important [27].Regarding protective immunity, nAb and antigen-specific memory T-cells are critical parameters.The nAb is a key player in the humoral immune responses that defends blocking viral entry into host cells and neutralizing their biological effect in collaboration with immune cells [8].Apart from the humoral immune responses, effective defense against SARS-CoV-2 can be achieved through cellular immune responses, including T cells and their related memory subsets [28].Thus, considering the successful prevention of viral infection and spreading, the type of vaccine, role of RBD Ab, and associations between humoral immune responses and cellular immune responses should be demonstrated.
In this study, we evaluated the qualitative and quantitative values among the RBD assay, nAb assay, and IGR assay in the vaccinated group.The values were compared based on the vaccine type, vector vaccine, MM vaccine, and mRNA vaccine, in a large population.In the qualitative and quantitative values of each assay, RBD Ab assays showed better performance in predicting the presence of nAb activity, but not the IGR level.The quantitative and qualitative values showed the lowest agreement and correlation in the vector vaccine group.The correlations between the RBD Ab assays and nAb assay were very strong in the vector vaccine, MM vaccine, and mRNA vaccine groups, and those between the RBD assays and IGR assay were very strong in only the mRNA vaccine group.
The virus neutralization test is considered the gold-standard method to quantify nAb values against SARS-CoV-2 [10].However, this method is time-consuming and labor-intensive, and requires biosafety laboratory level 3 facilities to handle risk group 3 pathogens such as SARS-CoV-2 [10].sVNT is a commercially available assay and showed a good correlation with the gold-standard method and the nAb titer [9,29].However, the TA values and correlation results were very different depending on the RBD assay platform, the number of vaccinations, and vaccine type [4,5].In this study, the RBD Ab assays and nAb assay showed moderate to strong agreement (kappa 0.76-0.83),and correlation coefficients were strong to very strong (ρ = 0.88-0.94).Similar to this study, a previous comparison study between the RBD Ab assays and nAb assays showed moderate to strong agreement (kappa 0.80-0.84)and a very strong correlation (correlation coefficient 0.89-0.90)[21].However, for international standardization to allow accurate calibration of assays and to use common units for data, the number of binding antibody units (BAU) per mL is used to compare assays detecting the same class of immunoglobulins with the same specificity [30].Since the conversion formula for each assay used a constant value suggested by the manufacturers, systemic bias might occur and affect the results.
We confirmed that the IGR values modestly increased after vaccination, but the agreement and correlation were relatively low between the IGR assay and other assays.In previous reports, the correlation between the RBD Ab and IGR values showed discrepancies [21,22].First, the cellular immune responses after vaccination greatly varied from individual to individual due to pre-existing cross-reactive memory T-cells against SARS-CoV-2 [13].Some studies have demonstrated that reactive T cells for SARS-CoV-2 are present in as many as 60% of individuals who have not been exposed to SARS-CoV-2.This implies that there might be a cross-reactive T-cell recognition between the coronavirus that causes the common cold and SARS-CoV-2 [14,31].
Regarding vaccine type, this study showed consistent findings with previous studies.The values of RBD Ab, nAb, and IGR in the mRNA vaccine group were higher than in vector and MM vaccines [4,[32][33][34].The qualitative and quantitative values were higher in mRNA with high Ab values than in vector and MM vaccines with relatively low Ab values.
In particular, the agreement and correlation between the RBD Ab assays and IGR assay and between the RBD Ab assays and nAb assay were higher in the mRNA vaccine than in the vector and MM vaccine groups.With the superb correlation, the higher the RBD Ab value, the better agreement and correlation with the nAb values [5,19,35].Thus, this study suggests the high potential of RBD Ab assays for qualitatively and quantitatively predicting IGR in an individual with the mRNA vaccine showing high values of RBD Ab.
This study has some limitations.First, our assay utilized a recombinant spike RBD based on the original SARS-CoV-2 strain.Thus, antibody binding avidity to recently emerged variants such as delta or omicron could be lower than the values we measured in this study.Second, this is a cross-sectional study.We do not consider the influence of Ab values decay dynamics according to time intervals.Despite these limitations, this study demonstrated that the RBD Ab assays showed good performance in predicting the presence of nAb activity.IGR level can be predicted by RBD assays only in the mRNA vaccine group.
In conclusion, different RBD Ab assays showed comparable performance and good correlation with the nAb assay across vaccine types.The RBD assays indicated superb agreement and correlation with IGR assay in the mRNA vaccine group with high Ab values.This study provides a better understanding of serological assays and the IGR assay for SARS-CoV-2 in the vaccinated group with a broad range of Ab values.

Diagnostics 2023 , 12 Figure 1 .
Figure 1.Quantitative correlation among the RBD Ab assays and nAb assay.(A) RBD Ab_A vs. RBD Ab_B, (B) RBD Ab_A vs. RBD Ab_R, and (C) RBD Ab_B vs. RBD Ab_R.The black line indicates the regression line of the vaccinated group.(D) RBD Ab_A, (grey triangle and dotted grey line) (E) RBD Ab_B (empty circle and solid grey line), and (F) RBD Ab_R (filled circle and solid black line) were compared with the nAb assay.Abbreviations: RBD Ab, receptor-binding domain antibodies; BAU, binding antibody units.

Figure 1 .
Figure 1.Quantitative correlation among the RBD Ab assays and nAb assay.(A) RBD Ab_A vs. RBD Ab_B, (B) RBD Ab_A vs. RBD Ab_R, and (C) RBD Ab_B vs. RBD Ab_R.The black line indicates the regression line of the vaccinated group.(D) RBD Ab_A, (grey triangle and dotted grey line) (E) RBD Ab_B (empty circle and solid grey line), and (F) RBD Ab_R (filled circle and solid black line) were compared with the nAb assay.Abbreviations: RBD Ab, receptor-binding domain antibodies; BAU, binding antibody units.

Figure 2 .
Figure 2. Quantitative correlation according to vaccine type among the RBD Ab assays and nAb assay.(A) RBD Ab_A vs. RBD Ab_B, (B) RBD Ab_A vs. RBD Ab_R, and (C) RBD Ab_B vs. RBD Ab_R were compared according to vaccine among BRD Ab assays.nAb assay was compared with (D) RBD Ab_A, (E) RBD Ab_B, and (F) RBD Ab_R.IGR assay was compared with (G) RBD Ab_A, (H) RBD Ab_B, (I) RBD Ab_R, and (J) the nAb assay.The regression lines were presented by the grey dotted line for vector vaccine, the grey solid for MM vaccines, and the black solid line for mRNA vaccine groups.

Figure 2 .
Figure 2. Quantitative correlation according to vaccine type among the RBD Ab assays and nAb assay.(A) RBD Ab_A vs. RBD Ab_B, (B) RBD Ab_A vs. RBD Ab_R, and (C) RBD Ab_B vs. RBD Ab_R were compared according to vaccine among BRD Ab assays.nAb assay was compared with (D) RBD Ab_A, (E) RBD Ab_B, and (F) RBD Ab_R.IGR assay was compared with (G) RBD Ab_A, (H) RBD Ab_B, (I) RBD Ab_R, and (J) the nAb assay.The regression lines were presented by the grey dotted line for vector vaccine, the grey solid for MM vaccines, and the black solid line for mRNA vaccine groups.

Table 1 .
Antibody and IGR levels for the RBD Ab assay, nAb assay, and IGR assay according to sample collection time and vaccine type.

Table 2 .
Concordance rates and agreements among the RBD Ab assays, nAb assay, and IGR assay.

Table 3 .
Concordance rates and agreement among RBD assays according to the vaccine types.

Table 3 .
Concordance rates and agreement among RBD assays according to the vaccine types.

Table 4 .
Mean difference of the values of antibody or interferon gamma release among RBD Ab_A, RBD Ab_B, RBD Ab_R, nAb, and IGR.