Validation of a Pseudovirus Neutralization Assay for Severe Acute Respiratory Syndrome Coronavirus 2: A High-Throughput Method for the Evaluation of Vaccine Immunogenicity

The evaluation of coronavirus disease 2019 (COVID-19) vaccine immunogenicity remains essential as the severe acute respiratory syncytial virus 2 (SARS-CoV-2) pandemic continues to evolve and as additional variants emerge. Neutralizing antibodies are a known correlate of protection for SARS-CoV-2 vaccines. A pseudovirus neutralization (PNT) assay was developed and validated at Novavax Clinical Immunology Laboratories to allow for the detection of neutralizing antibodies in vaccine clinical trial sera. The PNT assay was precise, accurate, linear, and specific in measuring SARS-CoV-2 neutralization titers in human serum for ancestral strain and the Omicron subvariants BA.5 and XBB.1.5, with an overall geometric coefficient of variation of ≤43.4%, a percent relative bias within the expected range of −60% to 150%, and a linearity value of R2 > 0.98 for all three strains. This pseudovirus assay will be useful for the analysis of vaccine clinical trial samples to assess vaccine immunogenicity. Future work will focus on modifying the assay for emerging variants, including XBB.1.16, EG.5.1, BA.2.86, and any other variants that emerge in the ongoing pandemic.


Introduction
The coronavirus disease 2019 (COVID-19) pandemic was caused by the severe acute respiratory syncytial virus 2 (SARS-CoV-2) virus [1].New variants continue to emerge and circulate, such as Omicron XBB.1.16,EG.5.1, HV.1, etc., leading to the sustained transmission of the virus and clinical infections.Earlier ancestral/prototype COVID-19 vaccines may not be as effective against variant SARS-CoV-2 because some variants (such as the Omicron subvariants) have developed immune evasion properties [2][3][4].There is a continued need for high-throughput, robust, cost-effective, and flexible methods that can be used to assess the immunogenicity of vaccines [5], especially with emerging variants and the continued use of updated vaccine boosters.
Pseudoviruses are engineered to contain genetic components of the original SARS-CoV-2 virus, but they lack the parts needed for sustained replication [9].This article describes the validation of an in vitro pseudovirus-based neutralization (PNT) assay to detect and quantitate neutralizing antibodies against SARS-CoV-2 ancestral and variant strains (Omicron BA.5 and Omicron XBB.1.5)at Novavax Clinical Immunology Laboratories.The assay was assessed for precision, specificity, linearity, and other validation parameters.The geometric mean titer (GMT) for the ancestral strain was normalized to the World Health Organization (WHO) international standard in international units (IU/mL) for the calibration and harmonization of the neutralization assay.
The PNT assay is a robust, high-throughput, and rapid method used for detecting neutralizing antibodies, which are a CoP for vaccine efficacy.This validated method is suitable for use in the testing of clinical trial sera.

Assay Procedure
A PNT assay was developed to assess virus-specific neutralizing antibodies against SARS-CoV-2 in human serum [10].To perform the assay, serum samples were heatinactivated in a 56 • C water bath for 30 min prior to the assay.Serum samples were serially diluted in an infection medium in a 96-well white opaque cell culture plate.The working dilution of the pseudovirus (eEnzyme, LLC, Gaithersburg, MD, USA) was then added to each well, before being incubated at 37 • C for 2 h.Then, HEK293T/hACE2 cells (obtained from Creative Biogene, Shirley, NY, USA) were added to the wells, before being incubated for 72 h at 37 • C.After incubation, the Bright-Glo™ luciferase substrate (Promega, Madison, WI, USA) was added to each well.Plates were incubated for 15 min at room temperature without ambient light.Viral entry into the cells was determined by measuring the luminescence with a SpectraMax ® iD3 microplate reader (Molecular Devices, San Jose, CA, USA).Neutralization titers represent the inhibitory dilution of serum samples at which relative luminescence units (RLUs) were reduced by 50% compared to virus control wells on each plate.Data were analyzed and 50% pseudovirus neutralization titers (ID 50 ) were calculated using 4-parameter curve fitting in SoftMax ® Pro software 7.1 (Molecular Devices, San Jose, CA, USA) (Figure 1).The assay was originally developed using the ancestral pseudovirus strain but was later modified for Omicron BA.
Positive quality control (QC) samples known to have high (HQC), medium (MQC), or low (LQC) PNT titers and negative controls (NCs) were either purchased from a commercial source (BioIVT) sera or prepared in house by mixing high-titer human serum with NC.
For the analysis of conversion to WHO international standard units, the WHO international standard (NIBSC code 20/136) was used in the ancestral strain PNT assay.

Precision
Precision was assessed using a panel of 40 serum samples that had a range of negative to high titers in the PNT assay.The PNT assay was performed in six runs by two analysts on three different days, with each sample tested twice per run.The percent geometric coefficient of variation (%GCV) was calculated based on variance component analysis using a sample as a fixed effect and the analyst and the day as random effects.The %GCV was then used to describe the precision (total, intra-assay, and inter-assay).

Specificity
The PNT assay was performed on samples collected prior to the SARS-CoV-2 pandemic, which were expected to cover the lower limit of quantification (LLoQ).Additionally, five pairs of pre-and post-vaccination sera from phase 3 trials of influenza or RSV vaccines (Novavax, Inc.) were assessed in the PNT assay.These sample pairs were expected to have <LLoQ PNT results while showing strong specific immune responses to influenza or RSV post-vaccination, demonstrating the specificity of the PNT assay.

Linearity
Two SARS-CoV-2 PNT-positive samples were tested in the PNT assay undiluted or at various dilutions in negative serum.The samples were tested twice in the same run in duplicates across six runs by two different analysts on three different days.Linear regression was then performed on the sample results and the percent relative bias was calculated as follows: % Relative Bias = 100 × Observed overall PNT GMT − Expected PNT Titer Expected PNT Titer (1)

Sensitivity and Analytical Range
The lowest and highest PNT titers that could be quantified with acceptable precision and accuracy (as above) were defined as the LLoQ and upper limit of quantitation (ULoQ), respectively.The LLoQ provides information about assay sensitivity.

Conversion to WHO International Standard Units (IU/mL)
To allow conversion between the assay units (PNT titers) and WHO international standard units (IU/mL), WHO international standard for the anti-SARS-CoV-2 immunoglobulin (human) ancestral strain (NIBSC code 20/136) was tested during validation.From the results, a conversion formula from PNT ID50 to IU/mL was defined.

Variant Assays
Although the PNT assay was first developed for the ancestral strain of SARS-CoV-2, the emergence of variants necessitated the adaptation of the assay for variant strains.The PNT assay was extended for the Omicron XBB.1.5 and Omicron BA.5 variants.A validation process similar to that of the ancestral strain was used for these variant assays.

Correlation Analyses
Serum samples were assessed in the PNT (with the same methodology used for validation) and live wild-type virus microneutralization (MN) assays [11].Linear regression and Bland-Altman assay analysis were conducted using GraphPad Prism ® software (San Diego, CA, USA; Version 9.3.1)and SAS software (v9.4,SAS Institute Inc., Cary, NC, USA) to determine the correlation between the PNT and MN assay results.
The specificity of the PNT assay was successfully demonstrated (Table 2).Human serum collected prior to the SARS-CoV-2 pandemic era had no detectable titer for the Wuhan D614 virus (<LLoQ).The LLoQ was determined based on the lowest titer values that were accurately and precisely detected for the sample, as mentioned in the Methods section (based on the acceptance criteria established in the validation protocol).The LLoQ represents the lowest quantity of the analyte that can be accurately, precisely, and reproducibly quantitated in the assay.The five pairs of pre-and post-vaccination sera from RSV-or influenza-vaccinated participants (pre-pandemic) all tested negative (<LLoQ) in the ancestral PNT assay.These samples had strong specific immune responses to influenza or RSV post-vaccination, demonstrating the specificity of the SARS-CoV-2 PNT assay.The linearity of the assay was successfully demonstrated, with an R 2 value of 0.983 for the individual positive samples that were examined (Figure 2 and Tables 3 and 4).0.983 N/A LCL, lower confidence limit; UCL, upper confidence limit.
The linearity of the assay was successfully demonstrated, with an R 2 value of 0.983 for the individual positive samples that were examined (Figure 2 and Tables 3 and 4).GCV, geometric coefficient of variation; GMT, geometric mean titer; PNT, pseudovirus neutralization. 1 Higher inter-assay %GCV for dilutions 2 and 1024 seems to be a random variation that might be biological in nature.
The LLoQ was assigned as a GMT of 42, based on the lowest titer values that were accurately and precisely detected for the sample (as well as the precision of the assay shown in Table 4).Based on the available tested samples, the ULoQ was found to be a titer of at least 14,863 for the ancestral strain.
The WHO international standard for anti-SARS-CoV-2 immunoglobulin (human) (NIBSC code: 20/136) was also tested during validation.The WHO-assigned concentration of 20/136 was 1000 IU/mL.The overall GMT of 20/136 from validation was 4680.9.Therefore, the conversion factor of 0.214 (1000/4680.9) was determined to be used as an internal reference for assay calibration and harmonization (PNT titer * 0.214 = IU/mL).

Assay Validations for Variants
The original assay was developed using the ancestral strain and later adapted to variant strains (including Omicron BA.5 and Omicron XBB.1.5).Quality control samples performed similarly for Omicron variants and the ancestral strain (Figure 3).The original assay was developed using the ancestral strain and later adapted to variant strains (including Omicron BA.5 and Omicron XBB.1.5).Quality control samples performed similarly for Omicron variants and the ancestral strain (Figure 3).Results for the assay validation parameters for the Omicron variants were similar to those for the ancestral strain (Supplementary Table S1).For Omicron BA.5, inter-assay, intra-assay, and total precision exhibited a %GCV ≤50% in 95%, 100%, and 97.5% of the samples, respectively.All pre-COVID healthy human samples were <LLoQ; all RSV-or influenza-vaccinated sera were <LLoQ for SARS-CoV-2 while showing strong specific responses to RSV or influenza.Linearity was successfully demonstrated (R 2 = 0.987 and 0.984), the LLoQ was defined as 36, and the ULoQ was determined to be at least 15,856.
For Omicron XBB.1.5,inter-assay, intra-assay, and total precision exhibited %GCV Results for the assay validation parameters for the Omicron variants were similar to those for the ancestral strain (Supplementary Table S1).For Omicron BA.5, inter-assay, intra-assay, and total precision exhibited a %GCV ≤ 50% in 95%, 100%, and 97.5% of the samples, respectively.All pre-COVID healthy human samples were <LLoQ; all RSV-or influenza-vaccinated sera were <LLoQ for SARS-CoV-2 while showing strong specific responses to RSV or influenza.Linearity was successfully demonstrated (R 2 = 0.987 and 0.984), the LLoQ was defined as 36, and the ULoQ was determined to be at least 15,856.

Correlation between the PNT Assay and the Whole-Virus-Based MN Assay
A systematic comparison between the whole wild-type-virus-based MN and PNT assays was performed.The PNT assay method used the same method as that used for validation (before validation).The data include all participants (both anti-NP-positive and -negative at baseline) at three time points (days 1, 14, and 28) with a total of N = 592 samples.A significant correlation of 50% neutralization titers between the two assays (N = 592, Pearson r = 0.83, p < 0.0001) was observed (Figure 4a).This is in line with the correlation reported in papers by Sholukh et al. [12] and Bhiman JN et al. [13] that reported a Pearson r = 0.81-0.89using similar cell-based whole-virus assays to draw comparisons with the PNT assay.In order to understand the bias between the PNT and MN assays, the Bland-Altman assay was performed (Figure 4b).This analysis revealed a small bias of 0.225 (95% limits of agreement from -0.54 to 0.98) and the majority of the data fell within two standard deviations of the mean, further confirming the initial correlation analysis.Our correlation results, along with the literature data, confirm the similarity between both the whole-virus and pseudovirus neutralization assays and therefore indicate no significant difference in the neutralizing antibody results (Figure 4a,b) from either of the assays.
Microorganisms 2024, 12, x FOR PEER REVIEW 9 of 12 the whole-virus and pseudovirus neutralization assays and therefore indicate no significant difference in the neutralizing antibody results (Figure 4a,b) from either of the assays.Our previous publication [10] was an FFP method paper, whereas this manuscript focuses on method validation.The principle of the assay mentioned in both manuscripts is the same.At the time of Cai Z et al.'s publication, the PNT assay was evolving, and we were still focused on developing/adapting the assay for other emerging variants in parallel (evaluating qualification data, etc.).Since Cai Z et al.'s publication, we have further fine-tuned the PNT methodology, leading to this validation effort.Some of the major differences between Cai Z et al.'s assay and the validated assay in this study include a longer assay duration (3 days in this study vs. 2 days in Cai Z et al.'s), a difference in the input virus (previously based on TCID50, but in this study, based on 50,000 RLU), and a difference in the backbone (lentivirus vs. the murine leukemia virus (MLV)).

Discussion
The PNT assay described herein showed accuracy, linearity, and specificity for SARS-CoV-2 neutralizing antibodies in human serum.The PNT assay was developed for the ancestral strain (Wuhan), as well as the Omicron subvariants BA.5 and XBB.1.5.As the IU/mL conversion factor was determined, results from the assay for the ancestral strain can also be converted to IU/mL, allowing comparisons to be drawn with other standardized assays and CoP levels, as well as comparisons with the data generated in other laboratories.These results suggest that the PNT assay will be useful for the measurement of neutralization in clinical trial samples for vaccine development.
The PNT assay provides a high-throughput, robust, and cost-effective method for the Our previous publication [10] was an FFP method paper, whereas this manuscript focuses on method validation.The principle of the assay mentioned in both manuscripts is the same.At the time of Cai Z et al.'s publication, the PNT assay was evolving, and we were still focused on developing/adapting the assay for other emerging variants in parallel (evaluating qualification data, etc.).Since Cai Z et al.'s publication, we have further fine-tuned the PNT methodology, leading to this validation effort.Some of the major differences between Cai Z et al.'s assay and the validated assay in this study include a longer assay duration (3 days in this study vs. 2 days in Cai Z et al.'s), a difference in the input virus (previously based on TCID50, but in this study, based on 50,000 RLU), and a difference in the backbone (lentivirus vs. the murine leukemia virus (MLV)).

Discussion
The PNT assay described herein showed accuracy, linearity, and specificity for SARS-CoV-2 neutralizing antibodies in human serum.The PNT assay was developed for the ancestral strain (Wuhan), as well as the Omicron subvariants BA.5 and XBB.1.5.As the IU/mL conversion factor was determined, results from the assay for the ancestral strain can also be converted to IU/mL, allowing comparisons to be drawn with other standardized assays and CoP levels, as well as comparisons with the data generated in other laboratories.These results suggest that the PNT assay will be useful for the measurement of neutralization in clinical trial samples for vaccine development.
The PNT assay provides a high-throughput, robust, and cost-effective method for the BSL-2 laboratory that can be used to assess neutralizing antibodies, which are a known correlate of protection [14].The pseudovirus assay was not only precise and accurate for the ancestral strain, but also for Omicron subvariants (BA.5 and XBB.1.5).This is critical as some Omicron subvariants have shown immune evasion properties [15], and the assessment of vaccine immunogenicity against these strains is important for determining the potential effectiveness of the vaccine with the updated strains.
Close similarity between whole virus-and pseudovirus-based neutralization assays has been well documented in the literature [12,16,17].Significant correlation (Pearson r = 0.83, p < 0.0001) and Bland-Altman analyses between the pseudovirus neutralization (conducted in a biosafety laboratory (BSL)-2) and whole-virus neutralization (conducted in BSL-3) assays confirm the concordance between the two assays.Our correlation data are in line with the correlation reported in Sholukh et al.'s [12] and Bhiman JN et al.'s [13] papers that reported Pearson r = 0.81-0.89using similar cell-based whole-virus assays to draw comparisons to the PNT assay.
The pseudovirus system employed in this PNT assay is particularly useful for clinical studies, CoP studies, and the assessment of emerging variants [9].This is, in part, because the PNT assay described is robust, high-throughput, and can be performed under less restrictive BSL conditions (BSL-2 compared to BSL-3).To our knowledge, this is the first report of a PNT assay for the XBB.1.5variant.As the XBB.1.5variant is currently circulating (at the time of writing), our validated assay will be useful for the evaluation of XBB.1.5strain-based monovalent vaccine boosters, recently recommended by the FDA [18] and the WHO [19].
In contrast to the ELISA-based measurements of either anti-rS antibodies or hACE2 binding inhibition (SARS-CoV-2 receptor binding inhibition) assays, this is a cell-based assay, involving additional assay set-up and incubation time, leading to a longer data turnaround time.Although the CoP levels for emerging variants using PNT still need to be established, it may be possible to determine them, considering that the CoP levels for the ancestral strain have already been established [14].As of now, the harmonization of pseudovirus assays for currently circulating XBB variants is difficult due to the current lack of suitable international standards for Omicron variants (XBB.1.5).
Compared to most of the laboratories, we used the MLV system for PNT in our assays.This assay system might be less affected by HIV antivirals (which might inhibit lentivirus based PNT assays and artificially show high neutralization titers) and might be more suitable for testing serum from subjects treated with HIV antivirals.We added this to the Discussion section.A limitation of this assay methodology is that the ULoQ was based on sufficiently high-titer serum samples available during the time of assay validation.Along similar lines, the adaptability of the validation method to forward-drifted variants is based on the availability of serum samples with reasonably high titers to establish the ULoQ.Other limitations of the PNT assay, already mentioned in our previous publication [10], include the need for special critical reagents (such as pseudoviruses expressing the S protein), the permissive cell lines, and the fact that the pseudovirus may not mimic the distribution/presentation of antigens presented on the live virus.

Conclusions
The PNT assay demonstrated accuracy, linearity, and specificity in measuring neutralization titers in human serum for the ancestral strain and the Omicron subvariants BA.5 and XBB.1.5.This pseudovirus assay will be a useful high-throughput and BSL-2-compatible method for analyzing vaccine clinical trial samples to assess vaccine immunogenicity.Future work will focus on adapting the assay for emerging variants, including XBB.1.16and any other variants that emerge.

Patents
The data reported in this manuscript are disclosed in a patent application.

12 Figure 2 .
Figure 2. Pseudovirus assay ancestral strain linearity of a positive sample.The positive sample was diluted in series and tested in six assay runs.Linearity was then evaluated by calculating the precision and accuracy of the titer at each dilution, and the slope of linear regression lines for each sample.Sample with only values above the LLoQ are presented in the figure.GMT, geometric mean titer; PNT, pseudovirus neutralization.

Figure 2 .
Figure 2. Pseudovirus assay ancestral strain linearity of a positive sample.The positive sample was diluted in series and tested in six assay runs.Linearity was then evaluated by calculating the precision and accuracy of the titer at each dilution, and the slope of linear regression lines for each sample.Sample with only values above the LLoQ are presented in the figure.GMT, geometric mean titer; PNT, pseudovirus neutralization.

Figure 3 .
Figure 3. Pseudovirus assay ancestral strain and variants' quality control standards performance.For (a) the ancestral strain, (b) the Omicron BA.5 variant, or (c) the Omicron XBB.1.5variant, positive quality control (QC) samples were tested in the pseudovirus assay to examine assay performance.* Values for NC that were <LLoQ were plotted as 0.5x LLoQ.GMT, geometric mean titer; HQC, positive QC samples with high PNT titers; LLoQ, lower limit of quantification; LQC, positive QC samples with low PNT titers; MQC, positive QC samples with medium PNT titers; NC, negative control; PNT, pseudovirus neutralization.

Figure 3 .
Figure 3. Pseudovirus assay ancestral strain and variants' quality control standards performance.For (a) the ancestral strain, (b) the Omicron BA.5 variant, or (c) the Omicron XBB.1.5variant, positive quality control (QC) samples were tested in the pseudovirus assay to examine assay performance.* Values for NC that were <LLoQ were plotted as 0.5x LLoQ.GMT, geometric mean titer; HQC, positive QC samples with high PNT titers; LLoQ, lower limit of quantification; LQC, positive QC samples with low PNT titers; MQC, positive QC samples with medium PNT titers; NC, negative control; PNT, pseudovirus neutralization.

Figure 4 .
Figure 4. Neutralization titers from whole wild-type virus (MN50) and pseudovirus (ID50)-based neutralization assays from N = 592 serum samples (including both anti-NP-positive and -negative) were subjected to correlation analysis for comparison purposes using GraphPad Prism (v9.3.1)(a) and Bland-Altman analysis (b).Pearson r values with a 95% confidence interval, R 2 values, and twotailed P values are shown in the graph.The solid line shows the regression trend and dotted lines show a 95% CI of the regression trend (a).In Figure (b), the red horizontal line shows a perfect agreement between the PNT and MN assays, the blue horizontal line shows the average log 10 difference between the PNT and MN assays, and the green horizontal lines show the average log10 difference ± 2 standard deviations.

Figure 4 .
Figure 4. Neutralization titers from whole wild-type virus (MN 50 ) and pseudovirus (ID50)-based neutralization assays from N = 592 serum samples (including both anti-NP-positive and -negative) were subjected to correlation analysis for comparison purposes using GraphPad Prism (v9.3.1)(a) and Bland-Altman analysis (b).Pearson r values with a 95% confidence interval, R 2 values, and two-tailed P values are shown in the graph.The solid line shows the regression trend and dotted lines show a 95% CI of the regression trend (a).In Figure (b), the red horizontal line shows a perfect agreement between the PNT and MN assays, the blue horizontal line shows the average log 10 difference between the PNT and MN assays, and the green horizontal lines show the average log10 difference ± 2 standard deviations.