Next Article in Journal
Enhanced Broad-Spectrum Efficacy of an L2-Based mRNA Vaccine Targeting HPV Types 6, 11, 16, 18, with Cross-Protection Against Multiple Additional High-Risk Types
Previous Article in Journal
Protective Impact of Influenza Vaccination on Healthcare Workers
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Vaccines
Volume 12
Issue 11
10.3390/vaccines12111238
vaccines-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Review

Evaluating the Quality of Studies Assessing COVID-19 Vaccine Neutralizing Antibody Immunogenicity

by
Maeva Katzmarzyk
1,
Robert Naughton
2,
Ioannis Sitaras
3,
Henning Jacobsen
1,
Melissa M. Higdon
4,* and
Maria Deloria Knoll
4
1
Department of Viral Immunology, Helmholtz Center for Infection Research, 38124 Braunschweig, Germany
2
Independent Researcher, London SW16 2TS, UK
3
W. Harry Feinstone Department of Molecular Microbiology and Immunology, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD 21205, USA
4
International Vaccine Access Center, Department of International Health, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD 21205, USA
*
Author to whom correspondence should be addressed.
Vaccines 2024, 12(11), 1238; https://doi.org/10.3390/vaccines12111238
Submission received: 17 September 2024 / Revised: 22 October 2024 / Accepted: 27 October 2024 / Published: 30 October 2024
Browse Figures
Versions Notes

Abstract

:
Objective: COVID-19 vaccine-neutralizing antibodies provide early data on potential vaccine effectiveness, but their usefulness depends on study reliability and reporting quality. Methods: We systematically evaluated 50 published post-vaccination neutralizing antibody studies for key parameters that determine study and data quality regarding sample size, SARS-CoV-2 infection, vaccination regimen, sample collection period, demographic characterization, clinical characterization, experimental protocol, live virus and pseudo-virus details, assay standardization, and data reporting. Each category was scored from very high to low or unclear quality, with the lowest score determining the overall study quality score. Results: None of the studies attained an overall high or very high score, 8% (n = 4) attained moderate, 42% (n = 21) low, and 50% (n = 25) unclear. The categories with the fewest studies assessed as ≥ high quality were SARS-CoV-2 infection (42%), sample size (30%), and assay standardization (14%). Overall quality was similar over time. No association between journal impact factor and quality score was found. Conclusions: We found that reporting in neutralization studies is widely incomplete, limiting their usefulness for downstream analyses.

1. Introduction

The COVID-19 pandemic is an ongoing threat to global health. With its emergence in late 2019, the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) spread globally, causing over 700 million COVID-19 infections and over 7 million deaths worldwide by January 2024 [1]. Vaccination campaigns, non-lethal infections, and public health strategies bolstered immunity and curtailed the virus’s spread. Continuously emerging variants, however, necessitate adaptations of authorized vaccines. Consequently, various vaccine platforms and new vaccine formulations that target new subvariants have been introduced into immunization programs and require ongoing performance evaluation. New products are authorized for use based on clinical trials assessing induced immunogenicity relative to products already in use. The clinical effectiveness of COVID-19 vaccines under current conditions can only be assessed by observational studies after they have been in widespread use. However, policy decisions and recommendations for vaccines against emerging variants are needed before robust clinical evidence becomes available. For timely decision-making, neutralizing antibody studies serve as a rapid surrogate for assessing vaccine efficacy.
Neutralizing antibody studies have been pivotal in understanding immunity induced by vaccination or infection, vaccine effectiveness, and viral susceptibility to neutralization [2,3,4,5,6]. As neutralizing antibodies have remained the principal correlate of protection against COVID-19 [7,8,9], these studies have the potential to directly inform vaccine policy. For example, data on immunogenicity played a key role in policy decisions around the introduction of booster vaccinations in the United States [10]. While neutralizing antibody studies continue to play a vitally important role in our understanding of COVID-19 vaccine performance, they are not without limitations. Although multiple types of in vitro neutralization assays have demonstrated robustness and comparability of results [11], several meta-analyses have demonstrated high heterogeneity between studies [2,12,13,14]. For example, in one meta-analysis, fold-reductions in neutralizing antibody titers against Omicron BA.1 relative to the index strain following first mRNA booster vaccination ranged from 1-fold to over 30-fold across studies [2]. The key concern in these studies lies in the lack of consistency, which results in heterogeneous study design and methods, and the execution of respective assays often varies between laboratories, limiting the comparability of study results. Methods reported often lack sufficient detail on aspects that could clarify reasons for obtaining different results. Despite the important role these studies have played in the current pandemic, to the best of our knowledge, there has been no assessment of the quality of these data with respect to comparability, robustness, and usefulness for downstream applications, like meta-analyses.
Neutralization results can be strongly affected by technical parameters that frequently vary across studies, including, but not limited to, “infectious virus input” and “time between infection and measurement” of infected cells in neutralization assays [15]. This emphasizes the importance of consistency and comprehensive reporting of details in neutralizing antibody studies. In the context of immune response and neutralization titers, the impact of non-technical parameters such as sample size, immunocompromising conditions of study participants, and time elapsed between infection/vaccination and sera collection will impact results. In an effort to reduce uncertainty in neutralization evidence to facilitate an improved understanding of vaccine performance under varying conditions, we created a Quality Assessment Tool (QAT) based on a previously published tool that includes key parameters likely affecting neutralization results [16].
The QAT enables a standardized, systematic, and objective evaluation of neutralizing antibody studies by assessing the overall quality and reliability of reported neutralization titers based on the information provided within each study. The tool assesses study characteristics, including methodological accuracy, experimental setting, characterization of cohorts, samples, and viruses, as well as completeness and quality of reporting. In addition to serving as an instrument to assess quality and enhance inter-study comparability, the QAT can serve as a checklist for the systematic conduct and reporting of neutralization studies.
To understand potential reasons for heterogeneous results across studies, we used the QAT to evaluate published manuscripts on post-vaccination neutralization response to SARS-CoV-2 variants to identify which factors responsible for limitations in comparability were most frequently observed. We then assessed whether there were improvements in study quality scores based on reporting and neutralization titer reliability over time by comparing studies published early after the COVID-19 vaccine introduction to studies published approximately two years later. The main aim of this study is to emphasize the importance of consistent conduct and reporting of neutralization studies to increase downstream utility and ensure a more effective response to infectious disease outbreaks.

2. Methods

2.1. Study Selection

Since the emergence of SARS-CoV-2 Omicron, we have continuously systematically reviewed post-vaccination neutralization studies for which methods have been previously described [13]. These studies are stored in a database consisting of around 500 studies at the start of the QAT analysis. We selected 50 peer-reviewed studies, the first 25 and last 25 published studies out of that database, with the 11th of July 2023 being the cutoff point. The first 25 studies were published between February and May 2021 [17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41]. The last 25 studies were published between March and July 2023 (Supplementary Table S1; [42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66]). Within the systematic literature review, studies that solely evaluated partial vaccination included samples solely collected from immunocompromised subjects or used variants of concern (e.g., Omicron) as the reference strain were excluded. Moreover, we excluded studies that used surrogate neutralization assays to evaluate neutralizing antibody titers and studies that reported NT80 or similar results instead of NT50. We solely included pseudo-virus studies that use reporter viruses and quantify infected cells via a luciferase fluorescent signal. Studies that report fold reductions in neutralization titers or studies that report data that enable the calculation of fold reduction in neutralization titers were included. All details of the search strategy and inclusion criteria can be derived from previous publications [2,12,13].

2.2. Quality Assessment Tool (QAT) Development and Update

We updated the previously published tool to incorporate booster vaccinations and breakthrough infections and to optimize additional parameters for clarity in the application [16] (Table 1; https://view-hub.org/sites/default/files/2024-09/QAT%20checklist%20view-hub_20240909.pdf (accessed on 28 October 2024)). Briefly, this tool employs 34 key questions, referred to as “parameters”, linked to eleven categories: sample size, SARS-CoV-2 infection, vaccination regimen, sample collection period, demographic characterization, clinical characterization, protocol, live virus, pseudo-virus, assay standardization, and data reporting. The parameters were identified through our experience in the design, conduct, and analysis of studies on neutralizing antibodies, in reviewing the literature on SARS-CoV-2 neutralization, and from discussions and collaboration with experts in this field. Answers to the parameters aim to determine the study’s quality in each category depending on the perceived extent of their effect on the comparability, reliability, and robustness of the reported neutralization titers, which is then translated into a qualitative assessment from “low” to “very high” quality. Briefly, each parameter is attributed to a specific response that carries an impact on the quality in the respective category (Supplementary Table S2). For example, a large sample size increases the quality, while a small sample size reduces quality. Some parameters with stratifications can have more complex outcomes. For example, the outcomes for the parameter “Was presence or absence of pre-vaccination infection confirmed?” are “Yes”, “No/ Not reported”, and “Not applicable”. Whereas outcomes for the parameter “Was any SARS-CoV-2 infection prior to completion of the primary vaccine regimen considered?” are “Yes—only naïve included”, “Yes—participants stratified by prior infection status”, “Reported but not considered”, and “Not reported”. When parameters are “not applicable” to the study, they have no impact on the quality.
An “unclear” rating was assigned when reporting for one or more parameters was insufficient to evaluate the quality based on that parameter. Unclear ratings contribute to the assessment of reporting quality, which we defined as completeness of information and was distinguished from the reliability assessment. In this context, under-reporting and missing details are indicators of low reporting quality. The overall quality of the study was determined by the lowest score obtained across all categories (“Very high” > “High” > “Moderate” > “Unclear” > “Low”). To regulate the impact of the quality scores of certain categories on the overall score, each category is attributed the maximum possible impact. The categories of sample size, SARS-CoV-2 infection, sample collection period, demographic and clinical characterization, and assay standardization can obtain the lowest possible quality score, “Low”, whereas vaccination regimen, protocol, live and pseudo-virus details, and data reporting can obtain an “Unclear” score as the lowest score as they have a lesser impact on the study quality. Due to this assessment being qualitative and based on provided data from the literature, we did not quantify the exact impact or uncertainty resulting from parameters with unclear or low-quality scores.

2.3. QAT Evaluation Process

To ensure unbiased evaluations, each study was randomly assigned to two of four experienced evaluators, who are authors of this manuscript, and independently assessed each study. Scoring discrepancies of paired assessments were discussed and resolved by the two assigned evaluators. Each study was evaluated based on the details provided within the publication.
The QAT evaluation process was managed using REDCap electronic data capture tools hosted at Johns Hopkins University [67,68].

2.4. Data Analysis

Within each category, we calculated the distribution of quality scores across studies. We compared the percentage of studies with very high or high scores vs. the remaining scores and the percentage with unclear quality ratings vs. the remaining scores for each category between studies published in 2021 and 2023 using chi-square or Fisher’s exact tests; the Wilcoxon rank-sum test was used to compare medians. The Student’s t-test was performed to assess differences in the average proportion of scores across studies between those published in 2021 vs. 2023. A Spearman correlation was used to assess the correlation between the studies’ overall quality scores and the impact factor of the journal the study was published in. Generation of graphs, tables, and statistical calculations were performed using Excel 2016 and SAS software 9.4.

3. Results

3.1. Evaluation of Reliability and Reporting Quality Across COVID-19 Studies

Almost all studies (44/50, 88%) had at least one category with unclear quality, and no study attained a high or very high overall quality score: 8% (n = 4) had moderate overall quality, 42% (n = 21) low and 50% (n = 25) unclear (Figure 1). The median percentage of categories with high or very high quality scores was 63.6% (IQR: 54.6–70.5%). Categories with the highest percentage of studies assigned a high or very high quality score included protocol (98%), data reporting (96%), sample collection period (78%), live virus details (74%), and vaccination regimen (68%; Figure 2); categories with the fewest studies assessed as very high or high quality were SARS-CoV-2 infection (42%), sample size (30%) and assay standardization (14%).
The median percentage of categories with an unclear quality score was 18.2% (IQR: 9.1–27.3%; Figure 1). Categories with the highest proportion of studies with unclear reporting were clinical characterization (56%), assay standardization (46%), and SARS-CoV-2 infection (30%; Figure 2). The most common issues were a lack of reporting of the health status of study participants (see parameters in the category “clinical characterization”), as well as underreporting of details on the neutralization assay, which includes reporting of the actual virus input in the assay and the use of a confirmation method if any. In addition, minor details on cell culture were frequently missing. Moreover, information on infections prior to vaccinations or breakthrough infections was often not considered. All studies reported sample size and sample collection period.

3.2. Temporal Trends in Reporting Quality and Reliability of COVID-19 Studies

Overall quality scores were similar between studies published in 2021 compared to 2023, with 4% (n = 1), 52% (n = 13), and 44% (n = 11) attaining moderate, low, and unclear quality scores, respectively, in 2021 compared to 12% (n = 3), 32% (n = 8), and 56% (n = 14) in 2023 (Figure 3). The median percentage of categories with a high or very high quality score (63.6% (IQR: 54.6–72.7%) vs. 54.5% (IQR: 54.6–63.6%), p = 0.26) or an unclear score (18.2% (IQR: 9.1–27.3%) vs. 18.2% (IQR: 9.1–18.2%), p = 0.25; calculated from Figure 1) was similar between 2021 and 2023 studies.
Quality scores improved between 2021 and 2023 for demographic characterization (36% vs. 76% high or very high quality, p = 0.004) and clinical characterization (24% vs. 56%, p = 0.02), as did the proportion with an unclear rating for these categories (demographic characterization: 32% vs. 4%, p = 0.02; clinical characterization: 72% vs. 40%, p = 0.02; Supplementary Figure S1). However, quality scores declined for vaccination regimen (88% vs. 48%, p = 0.005) and pseudo-virus (84% vs. 48%, p = 0.002; Figure 3). No statistically significant differences between periods were observed for other categories.
The 2021 studies were published in journals with a higher impact factor than the 2023 studies (median 50 (IQR: 28–56.2) vs. 10 (5.6–19.7), p < 0.0001). However, no meaningful correlation was found between the impact factor and the median percentage of categories with a high or very high rating (Spearman correlation [rs] = −0.01, p = 0.94) or the median percentage of categories with unclear scores (rs = 0.19, p = 0.18).

4. Discussion

Systematic application of a tool assessing quality as data reliability and reporting quality of studies evaluating SARS-CoV-2 antibody neutralization in response to COVID-19 vaccination identified notable limitations in published reports. Many of these limitations, which could be easily addressed, contribute to uncertainty surrounding heterogeneous findings and thus limit the usefulness of these studies. No study received an overall assessment of high or very high quality scores, but most studies provided precise details on the neutralization assay protocol, reported and confirmed the virus lineage of live virus strains used in the assay, used reasonable reference virus strains for calculating variant-specific fold-changes, and provided appropriate statistics and/or raw neutralization data. In addition, the majority of studies assessed here considered general factors affecting immune response in the study cohort, such as age, infecting variants from pre-vaccination or from breakthrough infections, and comparable sample collection periods after vaccination or infection.
We observed the lowest quality scores in the assay standardization category, which suggests low consideration of the impact of assay details on resulting neutralizing antibody titers. For example, reporting and confirmation of infectious virus input in the assay were predominantly neglected (see Supplementary Figure S2), yet previous studies have shown that these small details can greatly affect assay results [15,69]. In addition, since neutralization assays are cell-based, it is vital to report details on cell culture, including confluence, culture conditions, and, most importantly, maximum passage number. Quality in terms of reliability would also be improved by applying a higher standard for the assay details, including considering and confirming comparable infectious virus inputs by performing back titrations or using virus input controls and allowing an appropriate variance of titers.
When assessing reporting quality, we observed a general lack of information critical for evaluating quality across all assessed studies. Reporting of details on the study cohort, including prior infection history and health status, as well as details on the assay system used, which can strongly influence study results, were overlooked. Details like the health status of study participants, sequence confirmation of used (pseudo-) virus, virus input, and cell culture details, whilst known in respective labs, are often underestimated in their significance. Incorporating this information would enhance the study’s value for data comprehension.
When comparing studies from 2021 and 2023, we found no consistent improvement in reliability and reporting quality over time, as improvements in some categories were offset by declines in others. The lack of improvement could be partly attributed to sampling bias, which resulted in the inclusion of fewer studies with rigorous methods later in the pandemic. Reporting quality and reliability in the categories of demographic and clinical characterization improved from 2021 to 2023 despite the more complex epidemiological landscape in 2023, which was marked by diverse vaccine regimens and increased infection-induced immunity in populations. A greater experience with these neutralization studies over time might have improved the overall detail of reporting in these categories. In addition, in 2023, infection-derived immunity was more common and likely resulted in a greater emphasis on cohort descriptions. We recognize the critical role of reporting pre-vaccination infections, breakthrough infections, and infecting strains in the current intricate immunological landscape as essential for understanding resulting neutralizing antibody titers. Moreover, accurate demographic and clinical characterizations of the study cohort play a pivotal role in assessing the potential impact of age, sex, or health status on the immune response and, by extension, on neutralizing antibody titers within cohorts.
In contrast, studies published in 2021 provided greater details on vaccination regimens and pseudo-virus details than 2023 publications, probably because the first COVID-19 vaccines had recently started to circulate and felt more important to distinguish between different vaccines. Moreover, the parameters in the vaccination regimen category mainly include details on reporting and comparability of booster dosing intervals within the study cohort. As booster vaccinations were not common in 2021 compared to 2023, these parameters were not applicable and hence resulted in no reduction of quality in these studies, which could be an additional explanation for the differences between 2021 and 2023. Studies published later in the pandemic show frequent underreporting of details on the pseudo-virus system. Since it is known that single amino acid mutations in the spike of RNA viruses can alter the immune escape [15,70,71,72,73], reporting and sequence confirmation of spike mutations included in the pseudo-virus spike is crucial to evaluate the comparability of resulting neutralizing antibody titers. Recently published studies often referred to previous publications for details on the pseudo-virus system, which often did not provide the relevant information. Since pseudo-virus strains are typically obtained from reliable sources and virus alterations are unlikely, our tool rates the potential underreporting or lower data reliability in the pseudo-virus category as less influential.
Cochrane is a global network known for producing high-quality, evidence-based health research, including tools and frameworks for clinical trials and observational studies. Their Risk of Bias 2 (RoB 2) and Risk of Bias in Non-randomized Studies of Interventions (ROBINS-I) tools are widely used in systematic reviews and meta-analyses to assess the quality of individual studies [74,75]. These tools could be utilized to evaluate the bias in sample selection, measurement of bias, and reporting of bias from neutralizing antibody studies. However, they lack parameters like the variability in reagents (e.g., virus strains), differences in experimental setups (incubation times, cell lines for different assays), and technical issues that are highly relevant for neutralizing antibody studies. We consider a tool tailored to studies on neutralizing antibodies a valuable contribution to the field.
One limitation of this study is the lack of a gold standard against which to measure the validity of the QAT. While we believe the tool provides an objective, qualitative assessment of study quality with a focus on reliability and reporting quality, it should be used discerningly. Based on research findings, a group of experts discussed and agreed upon the parameters included in the QAT to impact neutralization results. However, a “low quality score” does not necessarily imply biased results; rather, it ensures that comparability between studies is limited and differences due to other factors cannot be ruled out. A more refined weighting of each parameter and category by its ability to influence results could optimize the tool further, but the benefit of the tool simply to guide conduct and reporting of neutralization studies details could broaden their usefulness. Another limitation of the study is the relatively small number of studies used for assessment; a larger number of studies conducted across diverse periods and settings would enhance representativeness. However, despite the small numbers, this review was sufficient to identify characteristics in studies in both early and recent time periods that could be improved.
This study provided insights that can guide the conduct and reporting of future neutralization studies, emphasizing the importance of consistent practices that can contribute to a more effective response to the COVID-19 pandemic and future infectious disease outbreaks. Neutralizing antibodies remain the primary correlate of protection against COVID-19, providing rapid insights into the neutralization escape of novel emerging SARS-CoV-2 variants or the performance of vaccine candidates. While individual studies may provide high-quality information, their power to inform policy is limited, which usually requires corroboration among several independent studies. Meta-analyses increase the robustness of evidence, but only when studies have high comparability and reliability, highlighting the importance of their systematic conduct and reporting. The application of checklists can enhance consistency, improve study reliability, and elucidate sources of heterogeneity between studies, thereby elevating the utility of such studies across various applications, particularly in informing policy. The substantial underreporting across various categories that this assessment identified, especially in cohort details and assay-related aspects, indicates a lack of consistent protocols and limited emphasis on reporting quality across laboratories and by reviewers for scientific journals. Through comprehensive and consistent reporting across laboratories, we can enhance the reliability and utility of neutralization studies and advance the quality of research in this critical area.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/vaccines12111238/s1, Table S1: Table including all 50 included neutralizing antibody studies published in 2021 and 2023, Table S2: Significance of each parameter present in the QAT is described. All possible outcomes and their respective impact on the quality score of each parameter is presented in the right columns; File S1: ASSAY DETAILS.

Author Contributions

H.J. and I.S. performed the literature review. H.J., M.K., R.N., I.S. and M.M.H. abstracted, analyzed, and interpreted all data. M.K., R.N., I.S. and H.J. performed the evaluation of studies using the QAT. M.M.H. performed statistical tests. M.K. wrote the manuscript. M.K. generated the figures. All authors read and revised the manuscript. M.D.K. and M.M.H. acquired funding. All authors have read and agreed to the published version of the manuscript.

Funding

This project was funded by the Coalition for Epidemic Preparedness Innovations, grant number 145310.

Data Availability Statement

The original contributions presented in the study are included in the article/Supplementary Materials. Further inquiries can be directed to the corresponding authors.

Conflicts of Interest

M.M.H. and M.D.K. report research grants from Pfizer (all paid to the institution) for unrelated projects. All other authors declare no competing interests.

References

  1. COVID-19 Cases|WHO COVID-19 Dashboard. Available online: https://data.who.int/dashboards/covid19/cases (accessed on 10 July 2024).
  2. Jacobsen, H.; Katzmarzyk, M.; Higdon, M.M.; Jiménez, V.C.; Sitaras, I.; Bar-Zeev, N.; Knoll, M.D. Post-Vaccination Neutralization Responses to Omicron Sub-Variants. Vaccines 2022, 10, 1757. [Google Scholar] [CrossRef]
  3. Zhan, X.-Y.; Chen, Y.; Zhang, X.; Shi, Q.; Chen, K.; Zeng, C.; Zhang, Y.; Liang, Y.; Li, W.; Li, M.; et al. Characterization of SARS-CoV-2-specific humoral immunity and associated factors in the healthy population post-vaccination. Vaccine 2024, 42, 175–185. [Google Scholar] [CrossRef] [PubMed]
  4. Zhong, Y.; Kang, A.Y.H.; Tay, C.J.X.; Li, H.E.; Elyana, N.; Tan, C.W.; Yap, W.C.; Lim, J.M.E.; Le Bert, N.; Chan, K.R.; et al. Correlates of protection against symptomatic SARS-CoV-2 in vaccinated children. Nat. Med. 2024, 30, 1373–1383. [Google Scholar] [CrossRef] [PubMed]
  5. McClelland, L.; Hare, R. The adsorption of influenza virus by red cells and a new in vitro method of measuring antibodies for influenza virus. Can. Public Health J. 1941, 32, 530–538. [Google Scholar]
  6. Donald, H.B.; Isaacs, A. Counts of influenza virus particles. J. Gen. Microbiol. 1954, 10, 457–464. [Google Scholar] [CrossRef] [PubMed]
  7. Hertz, T.; Levy, S.; Ostrovsky, D.; Oppenheimer, H.; Zismanov, S.; Kuzmina, A.; Friedman, L.M.; Trifkovic, S.; Brice, D.; Chun-Yang, L.; et al. Correlates of protection for booster doses of the SARS-CoV-2 vaccine BNT162b2. Nat. Commun. 2023, 14, 4575. [Google Scholar] [CrossRef]
  8. Khoury, D.S.; Cromer, D.; Reynaldi, A.; Schlub, T.E.; Wheatley, A.K.; Juno, J.A.; Subbarao, K.; Kent, S.J.; Triccas, J.A.; Davenport, M.P. Neutralizing antibody levels are highly predictive of immune protection from symptomatic SARS-CoV-2 infection. Nat. Med. 2021, 27, 1205–1211. [Google Scholar] [CrossRef]
  9. Regev-Yochay, G.; Lustig, Y.; Joseph, G.; Gilboa, M.; Barda, N.; Gens, I.; Indenbaum, V.; Halpern, O.; Katz-Likvornik, S.; Levin, T.; et al. Correlates of protection against COVID-19 infection and intensity of symptomatic disease in vaccinated individuals exposed to SARS-CoV-2 in households in Israel (ICoFS): A prospective cohort study. Lancet Microbe 2023, 4, e309–e318. [Google Scholar] [CrossRef]
  10. Mbaeyi, S.; Oliver, S.E.; Collins, J.P.; Godfrey, M.; Goswami, N.D.; Hadler, S.C.; Jones, J.; Moline, H.; Moulia, D.; Reddy, S.; et al. The Advisory Committee on Immunization Practices’ Interim Recommendations for Additional Primary and Booster Doses of COVID-19 Vaccines—United States, 2021. MMWR Morb. Mortal. Wkly. Rep. 2021, 70, 1545–1552. [Google Scholar] [CrossRef]
  11. Riepler, L.; Rössler, A.; Falch, A.; Volland, A.; Borena, W.; von Laer, D.; Kimpel, J. Comparison of Four SARS-CoV-2 Neutralization Assays. Vaccines 2020, 9, 13. [Google Scholar] [CrossRef]
  12. Jacobsen, H.; Sitaras, I.; Katzmarzyk, M.; Cobos Jiménez, V.; Naughton, R.; Higdon, M.M.; Deloria Knoll, M. Systematic review and meta-analysis of the factors affecting waning of post-vaccination neutralizing antibody responses against SARS-CoV-2. NPJ Vaccines 2023, 8, 159. [Google Scholar] [CrossRef] [PubMed]
  13. Sitaras, I.; Jacobsen, H.; Higdon, M.M.; Dowling, W.E.; Bar-Zeev, N.; Deloria Knoll, M. Systematic review of primary and booster COVID-19 sera neutralizing ability against SARS-CoV-2 omicron variant. NPJ Vaccines 2022, 7, 147. [Google Scholar] [CrossRef]
  14. VIEW-Hub by IVAC. Neutralization Studies|ViewHub. Available online: https://view-hub.org/vaccine/covid/neutralization-studies (accessed on 10 July 2024).
  15. Katzmarzyk, M.; Clesle, D.C.; van den Heuvel, J.; Hoffmann, M.; Garritsen, H.; Pöhlmann, S.; Jacobsen, H.; Čičin-Šain, L. Systematical assessment of the impact of single spike mutations of SARS-CoV-2 Omicron sub-variants on the neutralization capacity of post-vaccination sera. Front. Immunol. 2023, 14, 1288794. [Google Scholar] [CrossRef] [PubMed]
  16. Jacobsen, H.; Sitaras, I.; Jurgensmeyer, M.; Mulders, M.N.; Goldblatt, D.; Feikin, D.R.; Bar-Zeev, N.; Higdon, M.M.; Knoll, M.D. Assessing the Reliability of SARS-CoV-2 Neutralization Studies That Use Post-Vaccination Sera. Vaccines 2022, 10, 850. [Google Scholar] [CrossRef]
  17. Xie, X.; Liu, Y.; Liu, J.; Zhang, X.; Zou, J.; Fontes-Garfias, C.R.; Xia, H.; Swanson, K.A.; Cutler, M.; Cooper, D.; et al. Neutralization of SARS-CoV-2 spike 69/70 deletion, E484K and N501Y variants by BNT162b2 vaccine-elicited sera. Nat. Med. 2021, 27, 620–621. [Google Scholar] [CrossRef] [PubMed]
  18. Zhou, D.; Dejnirattisai, W.; Supasa, P.; Liu, C.; Mentzer, A.J.; Ginn, H.M.; Zhao, Y.; Duyvesteyn, H.M.E.; Tuekprakhon, A.; Nutalai, R.; et al. Evidence of escape of SARS-CoV-2 variant B.1.351 from natural and vaccine-induced sera. Cell 2021, 184, 2348–2361.e6. [Google Scholar] [CrossRef]
  19. Garcia-Beltran, W.F.; Lam, E.C.; St Denis, K.; Nitido, A.D.; Garcia, Z.H.; Hauser, B.M.; Feldman, J.; Pavlovic, M.N.; Gregory, D.J.; Poznansky, M.C.; et al. Multiple SARS-CoV-2 variants escape neutralization by vaccine-induced humoral immunity. Cell 2021, 184, 2372–2383.e9. [Google Scholar] [CrossRef]
  20. Wang, P.; Casner, R.G.; Nair, M.S.; Wang, M.; Yu, J.; Cerutti, G.; Liu, L.; Kwong, P.D.; Huang, Y.; Shapiro, L.; et al. Increased resistance of SARS-CoV-2 variant P.1 to antibody neutralization. Cell Host Microbe 2021, 29, 747–751.e4. [Google Scholar] [CrossRef]
  21. Collier, D.A.; de Marco, A.; Ferreira, I.A.T.M.; Meng, B.; Datir, R.P.; Walls, A.C.; Kemp, S.A.; Bassi, J.; Pinto, D.; Silacci-Fregni, C.; et al. Sensitivity of SARS-CoV-2 B.1.1.7 to mRNA vaccine-elicited antibodies. Nature 2021, 593, 136–141. [Google Scholar] [CrossRef]
  22. Muik, A.; Wallisch, A.-K.; Sänger, B.; Swanson, K.A.; Mühl, J.; Chen, W.; Cai, H.; Maurus, D.; Sarkar, R.; Türeci, Ö.; et al. Neutralization of SARS-CoV-2 lineage B.1.1.7 pseudovirus by BNT162b2 vaccine-elicited human sera. Science 2021, 371, 1152–1153. [Google Scholar] [CrossRef]
  23. Wu, K.; Werner, A.P.; Koch, M.; Choi, A.; Narayanan, E.; Stewart-Jones, G.B.E.; Colpitts, T.; Bennett, H.; Boyoglu-Barnum, S.; Shi, W.; et al. Serum Neutralizing Activity Elicited by mRNA-1273 Vaccine. N. Engl. J. Med. 2021, 384, 1468–1470. [Google Scholar] [CrossRef] [PubMed]
  24. Madhi, S.A.; Baillie, V.; Cutland, C.L.; Voysey, M.; Koen, A.L.; Fairlie, L.; Padayachee, S.D.; Dheda, K.; Barnabas, S.L.; Bhorat, Q.E.; et al. Efficacy of the ChAdOx1 nCoV-19 Covid-19 Vaccine against the B.1.351 Variant. N. Engl. J. Med. 2021, 384, 1885–1898. [Google Scholar] [CrossRef] [PubMed]
  25. Hoffmann, M.; Arora, P.; Groß, R.; Seidel, A.; Hörnich, B.F.; Hahn, A.S.; Krüger, N.; Graichen, L.; Hofmann-Winkler, H.; Kempf, A.; et al. SARS-CoV-2 variants B.1.351 and P.1 escape from neutralizing antibodies. Cell 2021, 184, 2384–2393.e12. [Google Scholar] [CrossRef] [PubMed]
  26. Kuzmina, A.; Khalaila, Y.; Voloshin, O.; Keren-Naus, A.; Boehm-Cohen, L.; Raviv, Y.; Shemer-Avni, Y.; Rosenberg, E.; Taube, R. SARS-CoV-2 spike variants exhibit differential infectivity and neutralization resistance to convalescent or post-vaccination sera. Cell Host Microbe 2021, 29, 522–528.e2. [Google Scholar] [CrossRef]
  27. Edara, V.V.; Norwood, C.; Floyd, K.; Lai, L.; Davis-Gardner, M.E.; Hudson, W.H.; Mantus, G.; Nyhoff, L.E.; Adelman, M.W.; Fineman, R.; et al. Infection- and vaccine-induced antibody binding and neutralization of the B.1.351 SARS-CoV-2 variant. Cell Host Microbe 2021, 29, 516–521.e3. [Google Scholar] [CrossRef] [PubMed]
  28. Stamatatos, L.; Czartoski, J.; Wan, Y.-H.; Homad, L.J.; Rubin, V.; Glantz, H.; Neradilek, M.; Seydoux, E.; Jennewein, M.F.; MacCamy, A.J.; et al. mRNA vaccination boosts cross-variant neutralizing antibodies elicited by SARS-CoV-2 infection. Science 2021, 372, 1413–1418. [Google Scholar] [CrossRef]
  29. Planas, D.; Bruel, T.; Grzelak, L.; Guivel-Benhassine, F.; Staropoli, I.; Porrot, F.; Planchais, C.; Buchrieser, J.; Rajah, M.M.; Bishop, E.; et al. Sensitivity of infectious SARS-CoV-2 B.1.1.7 and B.1.351 variants to neutralizing antibodies. Nat. Med. 2021, 27, 917–924. [Google Scholar] [CrossRef]
  30. Emary, K.R.W.; Golubchik, T.; Aley, P.K.; Ariani, C.V.; Angus, B.; Bibi, S.; Blane, B.; Bonsall, D.; Cicconi, P.; Charlton, S.; et al. Efficacy of ChAdOx1 nCoV-19 (AZD1222) vaccine against SARS-CoV-2 variant of concern 202012/01 (B.1.1.7): An exploratory analysis of a randomised controlled trial. Lancet 2021, 397, 1351–1362. [Google Scholar] [CrossRef]
  31. Dejnirattisai, W.; Zhou, D.; Supasa, P.; Liu, C.; Mentzer, A.J.; Ginn, H.M.; Zhao, Y.; Duyvesteyn, H.M.E.; Tuekprakhon, A.; Nutalai, R.; et al. Antibody evasion by the P.1 strain of SARS-CoV-2. Cell 2021, 184, 2939–2954.e9. [Google Scholar] [CrossRef]
  32. Shen, X.; Tang, H.; McDanal, C.; Wagh, K.; Fischer, W.; Theiler, J.; Yoon, H.; Li, D.; Haynes, B.F.; Sanders, K.O.; et al. SARS-CoV-2 variant B.1.1.7 is susceptible to neutralizing antibodies elicited by ancestral spike vaccines. Cell Host Microbe 2021, 29, 529–539.e3. [Google Scholar] [CrossRef]
  33. Wang, P.; Nair, M.S.; Liu, L.; Iketani, S.; Luo, Y.; Guo, Y.; Wang, M.; Yu, J.; Zhang, B.; Kwong, P.D.; et al. Antibody resistance of SARS-CoV-2 variants B.1.351 and B.1.1.7. Nature 2021, 593, 130–135. [Google Scholar] [CrossRef] [PubMed]
  34. Liu, Y.; Liu, J.; Xia, H.; Zhang, X.; Fontes-Garfias, C.R.; Swanson, K.A.; Cai, H.; Sarkar, R.; Chen, W.; Cutler, M.; et al. Neutralizing Activity of BNT162b2-Elicited Serum. N. Engl. J. Med. 2021, 384, 1466–1468. [Google Scholar] [CrossRef] [PubMed]
  35. Liu, Y.; Liu, J.; Xia, H.; Zhang, X.; Zou, J.; Fontes-Garfias, C.R.; Weaver, S.C.; Swanson, K.A.; Cai, H.; Sarkar, R.; et al. BNT162b2-Elicited Neutralization against New SARS-CoV-2 Spike Variants. N. Engl. J. Med. 2021, 385, 472–474. [Google Scholar] [CrossRef] [PubMed]
  36. Bates, T.A.; Leier, H.C.; Lyski, Z.L.; McBride, S.K.; Coulter, F.J.; Weinstein, J.B.; Goodman, J.R.; Lu, Z.; Siegel, S.A.R.; Sullivan, P.; et al. Neutralization of SARS-CoV-2 variants by convalescent and BNT162b2 vaccinated serum. Nat. Commun. 2021, 12, 5135. [Google Scholar] [CrossRef]
  37. Wang, G.-L.; Wang, Z.-Y.; Duan, L.-J.; Meng, Q.-C.; Jiang, M.-D.; Cao, J.; Yao, L.; Zhu, K.-L.; Cao, W.-C.; Ma, M.-J. Susceptibility of Circulating SARS-CoV-2 Variants to Neutralization. N. Engl. J. Med. 2021, 384, 2354–2356. [Google Scholar] [CrossRef]
  38. Geers, D.; Shamier, M.C.; Bogers, S.; den Hartog, G.; Gommers, L.; Nieuwkoop, N.N.; Schmitz, K.S.; Rijsbergen, L.C.; van Osch, J.A.T.; Dijkhuizen, E.; et al. SARS-CoV-2 variants of concern partially escape humoral but not T-cell responses in COVID-19 convalescent donors and vaccinees. Sci. Immunol. 2021, 6, eabj1750. [Google Scholar] [CrossRef]
  39. Anichini, G.; Terrosi, C.; Gori Savellini, G.; Gandolfo, C.; Franchi, F.; Cusi, M.G. Neutralizing Antibody Response of Vaccinees to SARS-CoV-2 Variants. Vaccines 2021, 9, 517. [Google Scholar] [CrossRef]
  40. Cao, Y.; Yisimayi, A.; Bai, Y.; Huang, W.; Li, X.; Zhang, Z.; Yuan, T.; An, R.; Wang, J.; Xiao, T.; et al. Humoral immune response to circulating SARS-CoV-2 variants elicited by inactivated and RBD-subunit vaccines. Cell Res. 2021, 31, 732–741. [Google Scholar] [CrossRef] [PubMed]
  41. Becker, M.; Dulovic, A.; Junker, D.; Ruetalo, N.; Kaiser, P.D.; Pinilla, Y.T.; Heinzel, C.; Haering, J.; Traenkle, B.; Wagner, T.R.; et al. Immune response to SARS-CoV-2 variants of concern in vaccinated individuals. Nat. Commun. 2021, 12, 3109. [Google Scholar] [CrossRef]
  42. Tan, N.H.; Geers, D.; Sablerolles, R.S.G.; Rietdijk, W.J.R.; Goorhuis, A.; Postma, D.F.; Visser, L.G.; Bogers, S.; van Dijk, L.L.A.; Gommers, L.; et al. Immunogenicity of bivalent omicron (BA.1) booster vaccination after different priming regimens in health-care workers in the Netherlands (SWITCH ON): Results from the direct boost group of an open-label, multicentre, randomised controlled trial. Lancet Infect. Dis. 2023, 23, 901–913. [Google Scholar] [CrossRef]
  43. Chen, J.-J.; Li, L.-B.; Peng, H.-H.; Tian, S.; Ji, B.; Shi, C.; Qian, C.; Jiang, W.-G.; Liu, M.-C.; Li, T.-T.; et al. Neutralization against XBB.1 and XBB.1.5 after omicron subvariants breakthrough infection or reinfection. Lancet Reg. Health West. Pac. 2023, 33, 100759. [Google Scholar] [CrossRef] [PubMed]
  44. Chen, X.; Ciric, C.; Gibson, T.; Anderson, L.J.; Anderson, E.J.; Rostad, C.A. Longitudinal Neutralizing and Functional Antibody Responses to Severe Acute Respiratory Syndrome Coronavirus 2 Variants Following Messenger RNA Coronavirus Disease 2019 Vaccination. Open Forum Infect. Dis. 2023, 10, ofad167. [Google Scholar] [CrossRef]
  45. Chen, Z.; Huang, T.; He, T.; Zha, G.; Zhu, Q.; Zhang, G.; Xiang, D.; Chen, M.; Li, H.; Ling, N.; et al. Humoral responses after primary and booster SARS-CoV-2 inactivated vaccination in patients with chronic hepatitis B virus infection: A longitudinal observational study. J. Med. Virol. 2023, 95, e28695. [Google Scholar] [CrossRef]
  46. Yamasoba, D.; Uriu, K.; Plianchaisuk, A.; Kosugi, Y.; Pan, L.; Zahradnik, J.; Ito, J.; Sato, K. Virological characteristics of the SARS-CoV-2 omicron XBB.1.16 variant. Lancet Infect. Dis. 2023, 23, 655–656. [Google Scholar] [CrossRef] [PubMed]
  47. Ren, W.; Zhang, Y.; Rao, J.; Wang, Z.; Lan, J.; Liu, K.; Zhang, X.; Hu, X.; Yang, C.; Zhong, G.; et al. Evolution of Immune Evasion and Host Range Expansion by the SARS-CoV-2 B.1.1.529 (Omicron) Variant. mBio 2023, 14, e0041623. [Google Scholar] [CrossRef] [PubMed]
  48. Guo, L.; Zhang, Q.; Zhong, J.; Chen, L.; Jiang, W.; Huang, T.; Li, Y.; Zhang, Y.; Xu, L.; Wang, X.; et al. Omicron BA.1 breakthrough infections in inactivated COVID-19 vaccine recipients induced distinct pattern of antibody and T cell responses to different Omicron sublineages. Emerg. Microbes Infect. 2023, 12, 2202263. [Google Scholar] [CrossRef] [PubMed]
  49. Liu, Y.; Wang, Z.; Zhuang, X.; Zhang, S.; Chen, Z.; Zou, Y.; Sheng, J.; Li, T.; Tai, W.; Yu, J.; et al. Inactivated vaccine-elicited potent antibodies can broadly neutralize SARS-CoV-2 circulating variants. Nat. Commun. 2023, 14, 2179. [Google Scholar] [CrossRef]
  50. Hernández, J.; Dehesa-Canseco, F.; Vázquez-López, A.B.; Reséndiz-Sandoval, M.; Caire-Juvera, G.; Solís-Hernández, M.; Valenzuela, O.; Gómez-Gil, B.; Mata-Haro, V. Neutralization of Omicron BA.1, BA.5.1.6, BQ.1.3 and XBB1.1 induced by heterologous vaccination Ad5-nCoV and mRNA-1273. Signal Transduct. Target. Ther. 2023, 8, 174. [Google Scholar] [CrossRef]
  51. Diem, G.; Jäger, M.; Dichtl, S.; Bauer, A.; Lass-Flörl, C.; Reindl, M.; Wilflingseder, D.; Posch, W. Vaccination and Omicron BA.1/BA.2 Convalescence Enhance Systemic but Not Mucosal Immunity against BA.4/5. Microbiol. Spectr. 2023, 11, e0516322. [Google Scholar] [CrossRef]
  52. Yamamoto, S.; Matsuda, K.; Maeda, K.; Oshiro, Y.; Inamura, N.; Mizoue, T.; Konishi, M.; Takeuchi, J.S.; Horii, K.; Ozeki, M.; et al. Omicron BA.1 neutralizing antibody response following Delta breakthrough infection compared with booster vaccination of BNT162b2. BMC Infect. Dis. 2023, 23, 282. [Google Scholar] [CrossRef]
  53. Cui, Z.; Luo, W.; Chen, R.; Li, Y.; Wang, Z.; Liu, Y.; Liu, S.; Feng, L.; Jia, Z.; Cheng, R.; et al. Comparing T- and B-cell responses to COVID-19 vaccines across varied immune backgrounds. Signal Transduct. Target. Ther. 2023, 8, 179. [Google Scholar] [CrossRef]
  54. Faraone, J.N.; Qu, P.; Evans, J.P.; Zheng, Y.-M.; Carlin, C.; Anghelina, M.; Stevens, P.; Fernandez, S.; Jones, D.; Lozanski, G.; et al. Neutralization escape of Omicron XBB, BR.2, and BA.2.3.20 subvariants. Cell Rep. Med. 2023, 4, 101049. [Google Scholar] [CrossRef]
  55. Carr, E.J.; Wu, M.Y.; Gahir, J.; Harvey, R.; Townsley, H.; Bailey, C.; Fowler, A.S.; Dowgier, G.; Hobbs, A.; Herman, L.; et al. Neutralising immunity to omicron sublineages BQ.1.1, XBB, and XBB.1.5 in healthy adults is boosted by bivalent BA.1-containing mRNA vaccination and previous Omicron infection. Lancet Infect. Dis. 2023, 23, 781–784. [Google Scholar] [CrossRef] [PubMed]
  56. Wang, F.; Huang, B.; Deng, Y.; Zhang, S.; Liu, X.; Wang, L.; Liu, Q.; Zhao, L.; Tang, L.; Wang, W.; et al. Neutralizing antibody levels associated with injectable and aerosolized Ad5-nCoV boosters and BA.2 infection. BMC Med. 2023, 21, 233. [Google Scholar] [CrossRef]
  57. Vikse, E.L.; Fossum, E.; Erdal, M.S.; Hungnes, O.; Bragstad, K. Poor neutralizing antibody responses against SARS-CoV-2 Omicron BQ.1.1 and XBB in Norway in October 2022. Influenza Other Respir. Viruses 2023, 17, e13144. [Google Scholar] [CrossRef]
  58. Toyoda, M.; Tan, T.S.; Motozono, C.; Barabona, G.; Yonekawa, A.; Shimono, N.; Minami, R.; Nagasaki, Y.; Miyashita, Y.; Oshiumi, H.; et al. Evaluation of Neutralizing Activity against Omicron Subvariants in BA.5 Breakthrough Infection and Three-Dose Vaccination Using a Novel Chemiluminescence-Based, Virus-Mediated Cytopathic Assay. Microbiol. Spectr. 2023, 11, e0066023. [Google Scholar] [CrossRef]
  59. Yang, J.; Hong, W.; Lei, H.; He, C.; Lei, W.; Zhou, Y.; Zhao, T.; Alu, A.; Ma, X.; Li, J.; et al. Low levels of neutralizing antibodies against XBB Omicron subvariants after BA.5 infection. Signal Transduct. Target. Ther. 2023, 8, 252. [Google Scholar] [CrossRef]
  60. Muangnoicharoen, S.; Wiangcharoen, R.; Nanthapisal, S.; Kamolratakul, S.; Lawpoolsri, S.; Jongkaewwattana, A.; Thitithanyanont, A.; Luvira, V.; Chinwangso, P.; Thanthamnu, N.; et al. Single Ad26.COV2.S booster dose following two doses of BBIBP-CorV vaccine against SARS-CoV-2 infection in adults: Day 28 results of a phase 1/2 open-label trial. Vaccine 2023, 41, 4648–4657. [Google Scholar] [CrossRef]
  61. Radion, E.I.; Mukhin, V.E.; Kholodova, A.V.; Vladimirov, I.S.; Alsaeva, D.Y.; Zhdanova, A.S.; Ulasova, N.Y.; Bulanova, N.V.; Makarov, V.V.; Keskinov, A.A.; et al. Functional Characteristics of Serum Anti-SARS-CoV-2 Antibodies against Delta and Omicron Variants after Vaccination with Sputnik V. Viruses 2023, 15, 1349. [Google Scholar] [CrossRef] [PubMed]
  62. Jin, P.-F.; Guo, X.-L.; Gou, J.-B.; Hou, L.-H.; Song, Z.-Z.; Zhu, T.; Pan, H.-X.; Zhu, J.-H.; Shi, F.-J.; Du, P.; et al. Immunogenicity and safety of heterologous immunisation with Ad5-nCOV in healthy adults aged 60 years and older primed with an inactivated SARS-CoV-2 vaccine (CoronaVac): A phase 4, randomised, observer-blind, non-inferiority trial. Lancet Reg. Health West. Pac. 2023, 38, 100829. [Google Scholar] [CrossRef] [PubMed]
  63. Dowell, A.C.; Lancaster, T.; Bruton, R.; Ireland, G.; Bentley, C.; Sylla, P.; Zuo, J.; Scott, S.; Jadir, A.; Begum, J.; et al. Immunological imprinting of humoral immunity to SARS-CoV-2 in children. Nat. Commun. 2023, 14, 3845. [Google Scholar] [CrossRef] [PubMed]
  64. Varasi, I.; Lai, A.; Fiaschi, L.; Bergna, A.; Gatti, A.; Caimi, B.; Biba, C.; Della Ventura, C.; Balotta, C.; Riva, A.; et al. Neutralizing antibodies response to novel SARS-CoV-2 omicron sublineages in long-term care facility residents after the fourth dose of monovalent BNT162b2 COVID-19 vaccination. J. Infect. 2023, 87, 270–272. [Google Scholar] [CrossRef] [PubMed]
  65. Wang, W.; Goguet, E.; Paz, S.; Vassell, R.; Pollett, S.; Mitre, E.; Weiss, C.D. Bivalent Coronavirus Disease 2019 Vaccine Antibody Responses to Omicron Variants Suggest That Responses to Divergent Variants Would Be Improved With Matched Vaccine Antigens. J. Infect. Dis. 2023, 228, 439–443. [Google Scholar] [CrossRef]
  66. Lyke, K.E.; Atmar, R.L.; Dominguez Islas, C.; Posavad, C.M.; Deming, M.E.; Branche, A.R.; Johnston, C.; El Sahly, H.M.; Edupuganti, S.; Mulligan, M.J.; et al. Immunogenicity of NVX-CoV2373 heterologous boost against SARS-CoV-2 variants. npj Vaccines 2023, 8, 98. [Google Scholar] [CrossRef]
  67. Harris, P.A.; Taylor, R.; Minor, B.L.; Elliott, V.; Fernandez, M.; O’Neal, L.; McLeod, L.; Delacqua, G.; Delacqua, F.; Kirby, J.; et al. The REDCap consortium: Building an international community of software platform partners. J. Biomed. Inform. 2019, 95, 103208. [Google Scholar] [CrossRef]
  68. Harris, P.A.; Taylor, R.; Thielke, R.; Payne, J.; Gonzalez, N.; Conde, J.G. Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support. J. Biomed. Inform. 2009, 42, 377–381. [Google Scholar] [CrossRef]
  69. Nie, J.; Li, Q.; Wu, J.; Zhao, C.; Hao, H.; Liu, H.; Zhang, L.; Nie, L.; Qin, H.; Wang, M.; et al. Establishment and validation of a pseudovirus neutralization assay for SARS-CoV-2. Emerg. Microbes Infect. 2020, 9, 680–686. [Google Scholar] [CrossRef]
  70. Pastorio, C.; Noettger, S.; Nchioua, R.; Zech, F.; Sparrer, K.M.J.; Kirchhoff, F. Impact of mutations defining SARS-CoV-2 Omicron subvariants BA.2.12.1 and BA.4/5 on Spike function and neutralization. iScience 2023, 26, 108299. [Google Scholar] [CrossRef] [PubMed]
  71. Smith, D.J.; Lapedes, A.S.; De-Jong, J.C.; Bestebroer, T.M.; Jones, T.C.; Rimmelzwaan, G.F.; Osterhaus, A.D.M.; Fouchier, R.A.M. Mutations, Drift, and the Influenza Archipelago. Discov. Med. 2009, 4, 371–377. [Google Scholar]
  72. Smith, D.J.; Lapedes, A.S.; de Jong, J.C.; Bestebroer, T.M.; Rimmelzwaan, G.F.; Osterhaus, A.D.M.E.; Fouchier, R.A.M. Mapping the antigenic and genetic evolution of influenza virus. Science 2004, 305, 371–376. [Google Scholar] [CrossRef]
  73. Yamanaka, T.; Cullinane, A.; Gildea, S.; Bannai, H.; Nemoto, M.; Tsujimura, K.; Kondo, T.; Matsumura, T. The potential impact of a single amino-acid substitution on the efficacy of equine influenza vaccines. Equine Vet. J. 2015, 47, 456–462. [Google Scholar] [CrossRef] [PubMed]
  74. Sterne, J.A.; Hernán, M.A.; Reeves, B.C.; Savović, J.; Berkman, N.D.; Viswanathan, M.; Henry, D.; Altman, D.G.; Ansari, M.T.; Boutron, I.; et al. ROBINS-I: A tool for assessing risk of bias in non-randomised studies of interventions. BMJ 2016, 355, i4919. [Google Scholar] [CrossRef] [PubMed]
  75. Sterne, J.A.C.; Savović, J.; Page, M.J.; Elbers, R.G.; Blencowe, N.S.; Boutron, I.; Cates, C.J.; Cheng, H.-Y.; Corbett, M.S.; Eldridge, S.M.; et al. RoB 2: A revised tool for assessing risk of bias in randomised trials. BMJ 2019, 366, l4898. [Google Scholar] [CrossRef] [PubMed]
Vaccines 12 01238 g001
Figure 1. Individual quality scores for each category of all assessed studies. Studies are sorted for impact factor and publication year. References [17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66] to the respective study number is provided in Supplementary Table S1. Quality scores from low to very high are provided for eleven categories. The maximum impact is “unclear” for the categories of vaccination regimen, protocol, live virus, pseudo-virus, and data reporting; all remaining categories have “low” as the lowest quality score. Overall quality is assigned the lowest score observed across categories. For each study, the percentage of the 11 categories with high or very high quality scores and unclear scores is provided. 1 Unclear quality is specified as no sufficient information to evaluate the quality of the data, indicating low reporting quality.
Figure 1. Individual quality scores for each category of all assessed studies. Studies are sorted for impact factor and publication year. References [17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66] to the respective study number is provided in Supplementary Table S1. Quality scores from low to very high are provided for eleven categories. The maximum impact is “unclear” for the categories of vaccination regimen, protocol, live virus, pseudo-virus, and data reporting; all remaining categories have “low” as the lowest quality score. Overall quality is assigned the lowest score observed across categories. For each study, the percentage of the 11 categories with high or very high quality scores and unclear scores is provided. 1 Unclear quality is specified as no sufficient information to evaluate the quality of the data, indicating low reporting quality.
Vaccines 12 01238 g001
Vaccines 12 01238 g002
Figure 2. Percentage of studies receiving specific quality scores for each category of the QAT. Quality ratings from low to very high quality scores on the x-axis. If the maximum possible impact of a category is unclear, low quality is not applicable (n.a.) for that category. The percentage of studies with the respective quality scores is provided above the bars.
Figure 2. Percentage of studies receiving specific quality scores for each category of the QAT. Quality ratings from low to very high quality scores on the x-axis. If the maximum possible impact of a category is unclear, low quality is not applicable (n.a.) for that category. The percentage of studies with the respective quality scores is provided above the bars.
Vaccines 12 01238 g002
Vaccines 12 01238 g003
Figure 3. Comparison of quality scores for each category of studies published in 2021 vs. 2023. The percentage of studies with very high, high, moderate, unclear, and low quality scores are shown by publication year (February–May 2021 and March–July 2023) for 25 studies in each period. p-values for comparisons of very high or high vs. other categories are provided for each category. x The maximum impact is “unclear” for the categories of vaccination regimen, protocol, live virus, pseudo-virus, and data reporting; all remaining categories have “low” as the maximum quality score. Overall quality is assigned the lowest score observed across categories. 1 Unclear quality is specified as no sufficient information to evaluate the quality of the data, indicating low reporting quality. Statistical analysis was performed by Student’s t-test. Statistical significance was defined by a value of * <0.05; ** <0.01; ns, not significant.
Figure 3. Comparison of quality scores for each category of studies published in 2021 vs. 2023. The percentage of studies with very high, high, moderate, unclear, and low quality scores are shown by publication year (February–May 2021 and March–July 2023) for 25 studies in each period. p-values for comparisons of very high or high vs. other categories are provided for each category. x The maximum impact is “unclear” for the categories of vaccination regimen, protocol, live virus, pseudo-virus, and data reporting; all remaining categories have “low” as the maximum quality score. Overall quality is assigned the lowest score observed across categories. 1 Unclear quality is specified as no sufficient information to evaluate the quality of the data, indicating low reporting quality. Statistical analysis was performed by Student’s t-test. Statistical significance was defined by a value of * <0.05; ** <0.01; ns, not significant.
Vaccines 12 01238 g003
Table 1. Parameters of each category assessed in the QAT.
Table 1. Parameters of each category assessed in the QAT.
CategoriesParameters
1Sample size1.1How many samples were included?
2SARS-CoV-2 infection2.1Was any SARS-CoV-2 infection prior to completion of the primary vaccine regimen considered?
2.2Was presence or absence of pre-vaccination infection confirmed?
2.3Were breakthrough infections considered in the study cohort?
2.4Were breakthrough infections confirmed?
2.5Were infection-naïve/previously infected/ breakthrough infected samples stratified in the analyses?
3Vaccination regimen3.1Do the authors report booster dosing interval?
3.2Are the booster dosing intervals comparable?
3.3Do the authors stratify for partial and complete primary regimen?
4Sample collection period4.1Were all samples taken at least seven days post last immunogenic event?
4.2Are the results stratified, OR are all samples taken ≥ two weeks and ≤ 4 months post-last immunogenic event?
5Demographic characterization5.1Is the age distribution of all subjects reported?
5.2Are results stratified by age group?
5.3Is the sex distribution of all participants reported?
5.4If only a subgroup of the initial study cohort was analyzed, did the cohort selection happen unbiased?
5.5Was the infecting variant/ or variant prevalence reported?
5.6Was the study period and geographic location reported?
5.7If (multiple) breakthrough infections occurred, were the results stratified for the infecting variant(s)?
6Clinical characterization6.1Is any relevant clinical characterization reported?
6.2Are the results stratified for immunocompromised?
7Protocol7.1Is the precise assay type and endpoint reported?
7.2Do the authors provide a precise protocol for the neutralization assay within the manuscript?
8Live virus8.1Is the virus lineage reported?
8.2Has the sequence been confirmed by sequencing?
9Pseudo-virus9.1Are the construct details reported?
9.2Are all variant-associated spike mutations included in the pseudo-virus?
9.3Has the sequence been confirmed by sequencing?
10Assay standardization10.1Is the amount of infectious virus input (virus titer) used for neutralization assays reported, and if so, are they consistent and have a small input variance?
10.2Was the intended virus titer used for neutralization assays confirmed by back-titration or virus controls?
10.3Are precise details on cell culture reported?
11Data reporting11.1Is the raw data for neutralization titers reported?
11.2Is the reference virus used for calculating variant-specific fold changes reasonable?
11.3Are appropriate statistics provided?
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Katzmarzyk, M.; Naughton, R.; Sitaras, I.; Jacobsen, H.; Higdon, M.M.; Deloria Knoll, M. Evaluating the Quality of Studies Assessing COVID-19 Vaccine Neutralizing Antibody Immunogenicity. Vaccines 2024, 12, 1238. https://doi.org/10.3390/vaccines12111238

AMA Style

Katzmarzyk M, Naughton R, Sitaras I, Jacobsen H, Higdon MM, Deloria Knoll M. Evaluating the Quality of Studies Assessing COVID-19 Vaccine Neutralizing Antibody Immunogenicity. Vaccines. 2024; 12(11):1238. https://doi.org/10.3390/vaccines12111238

Chicago/Turabian Style

Katzmarzyk, Maeva, Robert Naughton, Ioannis Sitaras, Henning Jacobsen, Melissa M. Higdon, and Maria Deloria Knoll. 2024. "Evaluating the Quality of Studies Assessing COVID-19 Vaccine Neutralizing Antibody Immunogenicity" Vaccines 12, no. 11: 1238. https://doi.org/10.3390/vaccines12111238

APA Style

Katzmarzyk, M., Naughton, R., Sitaras, I., Jacobsen, H., Higdon, M. M., & Deloria Knoll, M. (2024). Evaluating the Quality of Studies Assessing COVID-19 Vaccine Neutralizing Antibody Immunogenicity. Vaccines, 12(11), 1238. https://doi.org/10.3390/vaccines12111238

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop