Effectiveness of BNT162b2 and mRNA-1273 Vaccines against COVID-19 Infection: A Meta-Analysis of Test-Negative Design Studies

Although numerous COVID-19 vaccines are effective against COVID-19 infection and variants of concern (VOC) in the real world, it is imperative to obtain evidence of the corresponding vaccine effectiveness (VE). This study estimates the real-world effectiveness of the BNT162b2 and mRNA-1273 vaccines against COVID-19 infection and determines the influence of different virus variants on VE by using test-negative design (TND) studies. We systematically searched for published articles on the efficacy of BNT162b2 and mRNA-1273 against COVID-19 infection. Two researchers independently selected and extracted data from eligible studies. We calculated the VE associated with different vaccine types, SARS-CoV-2 variants, and vaccination statuses, using an inverse variance random-effects model. We selected 19 eligible studies in the meta-analysis from 1651 records. For the partially vaccinated group, the VE of BNT162b2 and mRNA-1273 was 61% and 78% against COVID-19 infection, respectively. For the completely vaccinated group, the VE of BNT162b2 and mRNA-1273 was 90% and 92% against COVID-19 infection, respectively. During subgroup analyses, the overall VE of BNT162b2 and mRNA-1273 against the Delta variant was 53% and 71%, respectively, for the partially vaccinated group; the respective VE values were 85% and 91% for the fully vaccinated group. Irrespective of the BNT162b2 or mRNA-1273 vaccines, the Delta variant significantly weakened vaccine protection for the partially vaccinated group, while full vaccination was highly effective against COVID-19 infection and various VOC. The mRNA-1273 vaccine is more effective against COVID-19 infection and VOC than the BNT162b2 vaccine, especially for the partially vaccinated group. Overall, the results provide recommendations for national and regional vaccine policies.


Introduction
The global spread of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has resulted in more than 255 million confirmed cases and more than 5 million deaths as of 19 November 2021. In addition, the global COVID-19 situation remains severe, with the number of new novel coronavirus cases going up 0.5 million per day. The advent of COVID-19 vaccines and widespread vaccination across the globe have offered glimmers of hope for the eradication of COVID-19, and more than 7.3 billion dose of vaccine have been Vaccines 2022, 10, 469 2 of 17 administered as of 17 November 2021 [1]. Both BNT162b2 and mRNA-1273 vaccines have been shown to be highly effective in multiple randomized clinical trials (RCTS) [2][3][4][5][6].
However, clinical trials may not simulate the real-world mass vaccination of COVID-19 vaccines because these studies were conducted in highly controlled environments. In contrast, the emerging variants defined as variants of concern (VOC), including the Alpha, Beta, Gamma, and Delta variants, were associated with increased transmission, more severe outcomes, reduced antibody neutralization from previous infection or vaccination, and lower vaccine effectiveness [7][8][9][10]. Thus, it is urgent and necessary to synthesize evidence of the vaccine effectiveness (VE) of COVID-19 vaccines against COVID-19 infection and VOC in the real world.
Because of feasibility and ethical issues, observational studies will replace randomized controlled trials as the preferred method of assessing the efficacy of COVID-19 vaccines. Test-negative design-one of the main methods for assessing the effectiveness of influenza vaccines-has increasingly been used by researchers as a convenient and effective way to assess the efficacy of COVID-19 because it reduces the bias due to infection misclassification and confusion in care-seeking behavior compared to other observational studies [11][12][13][14]. More importantly, when a large randomized controlled trial is impossible, the design can detect changes in vaccine effectiveness or measure VE against VOC [15].
Global mass vaccination requires a synthesis of evidence on SARS-CoV-2 VE to drive global health policy strategies. Especially, the VE against VOC may influence the choices of policy makers with regard to vaccine supply, procurement, deployment, and vaccination. We conducted a meta-analysis using TND studies to assess the VE of BNT162b2 and mRNA-1273 vaccines against COVID-19 infection and VOC.

Data Sources and Search Strategies
We systematically searched for articles published from 2020, when the first vaccine against COVID-19 was released, to October 26, 2021, in the following electronic databases: PubMed, Cochrane Library, Web of Science, Embase, Scopus, and preprint servers (Biorxiv and Medrxiv). We employed literature management with EndNote-X9. This meta-analysis used the following combinations as search terms: "SARS-CoV-2, COVID-19, 2019nCoV, Pfizer, BNT162b2, Comirnaty, mRNA-1273, Spikevax, Moderna, mRNA vaccine, effectiveness, efficacy, test-negative case-control, test-negative design." Terms could appear in either the title or abstract of the article. There were no restrictions on the language of the articles that were reviewed, and the reference lists of related articles were also browsed to retrieve additional studies. Results were screened independently by two reviewers, and disputes were resolved in consultation with a third reviewer. Only TND studies evaluating the effectiveness of BNT162b2 or mRNA-1273 vaccine against SARS-CoV-2 infection were included. Studies eligible for inclusion should meet the following criteria: definition of case is people testing positive for SARS-CoV-2 confirmed by PCR and control is people testing negative for SARS-CoV-2 confirmed by PCR [14]; the efficacies of BNT162b2 and mRNA-1273 vaccines were reported separately, and measurements used to assess the VE were adjusted. This meta-analysis excluded studies if they contained exclusively mRNA vaccine VE data (or mixed BNT162b2 and mRNA-1273 VE data which could not be separated). Studies where the outcome was not COVID-19 infection, such as hospitalization, severe illness, death, and other severe outcomes, were excluded.

Data Extraction and Study Quality Assessment
Two research personnel extracted data independently from selected studies, and a third author was consulted when consensus was lacking. The following data from selected articles were extracted: first author's surname, country, sample, total number of participants, number of vaccinated and unvaccinated individuals in the cases and controls, type of vaccine, variants of COVID-19, adjusted VE estimate, and adjustment factors. We only extracted data related to SARS-CoV-2 infection when both the results of SARS-CoV-2 infection and symptomatic infection were reported. Quality assessment of researches was conducted with the Newcastle-Ottawa scale (NOS) [16] by a researcher independently, and the second researcher was responsible for inspection and verification. The score >7 was considered high quality, 5-6 was medium, <5 was low, and a study with a score of 5 or greater was included in the meta-analysis.

Data Synthesis and Analysis
Relevant study characteristics were synthesized in a tabular format. VE against SARS-CoV-2 infection and VOC with different vaccination situations were evaluated with the odds rate (OR) of vaccination between cases and controls and were calculated using the following formula: VE = (1 − OR) × 100%. One-dose vaccination and two-dose vaccination were defined as partial and full vaccination, respectively. Meta-analysis with the randomeffects model was employed to calculate the pooled OR and 95% confidence intervals, and statistical heterogeneity between the studies was assessed using I 2 . We performed subgroup analysis stratifying by different times after vaccination, VOC, and vaccine situation. All analyses were conducted using Stata 11.2 (StataCorp, College Station, TX, USA). A twotailed p value < 0.05 was considered to be statistically significant.

Data Extraction and Study Quality Assessment
Two research personnel extracted data independently from selected studies, and a third author was consulted when consensus was lacking. The following data from selected articles were extracted: first author's surname, country, sample, total number of participants, number of vaccinated and unvaccinated individuals in the cases and controls, type of vaccine, variants of COVID-19, adjusted VE estimate, and adjustment factors. We only extracted data related to SARS-CoV-2 infection when both the results of SARS-CoV-2 infection and symptomatic infection were reported. Quality assessment of researches was conducted with the Newcastle-Ottawa scale (NOS) [16] by a researcher independently, and the second researcher was responsible for inspection and verification. The score >7 was considered high quality, 5-6 was medium, <5 was low, and a study with a score of 5 or greater was included in the meta-analysis.

Data Synthesis and Analysis
Relevant study characteristics were synthesized in a tabular format. VE against SARS-CoV-2 infection and VOC with different vaccination situations were evaluated with the odds rate (OR) of vaccination between cases and controls and were calculated using the following formula: VE = (1 − OR) × 100%. One-dose vaccination and two-dose vaccination were defined as partial and full vaccination, respectively. Meta-analysis with the random-effects model was employed to calculate the pooled OR and 95% confidence intervals, and statistical heterogeneity between the studies was assessed using I 2 . We performed subgroup analysis stratifying by different times after vaccination, VOC, and vaccine situation. All analyses were conducted using Stata 11.2 (StataCorp, College Station, TX, USA). A two-tailed p value < 0.05 was considered to be statistically significant.

Pooled VE against VOC
To assess the vaccine effectiveness of BNT162b2 and mRNA-1273 vaccines against VOC, we performed a subgroup analysis for VOC in partially vaccinated and fully vaccinated individuals of BNT162b2 and mRNA-1273 vaccine. In the partial vaccination

Discussion
In this meta-analysis of the VE against COVID-19 infection and VOC from TND studies, in both BNT162b2 and mRNA-1273, partial vaccination had low VE against COVID-19 infection with VE of 61% and 78%, respectively, and full vaccination had significantly higher efficacy against COVID-19 infection than the partial vaccine with VE of 90% and 92%, respectively, which was consistent with, but slightly less effective than the phase 3 randomized controlled trials of BNT162b2 [5] and mRNA-1273 [4] vaccines. The differences in vaccine effectiveness between clinical trials and real-world studies may result from differences in the definition of COVID-19. In clinical trials, the presence of symptoms and a positive SARS-CoV-2 RT-PCR test was defined as confirmed COVID-19, whereas in the studies included in our meta-analyses, only a positive SARS-CoV-2 RT-PCR test confirmed COVID-19. In addition, some studies have reported that vaccines are less effective in protecting older adults against COVID-19 infection, and four articles (Skowronski et al. [18], Thompson et al. [21], Bernal et al. [27] and Kissling et al. [33]) included in our analysis studied vaccine effectiveness in adults over the age of 50, which could reduce the overall vaccine protection. Furthermore, a low sensitivity or specificity of PCR tests in TND may lead to misclassification of cases and controls, which would also weaken estimates of vaccine effectiveness [20].
Similar to the results of the vaccine for preventing infection, full vaccination was significantly more effective for preventing hospitalization than partial vaccination. The difference is that the effectiveness of vaccines in preventing hospitalization is slightly higher than that of preventing infection under the same vaccine status (Table A3, Figure A1). Our findings also showed that partial vaccination of both BNT162b2 and mRNA-1273 vaccines significantly reduced the effect on Delta compared with the Alpha variant, but the difference was significantly reduced after full vaccination. The same phenomenon was observed in two U.S. CDC reports [36,37] and an observational study on the efficacy of vaccines against Delta infection [38]. In this analysis, the estimated vaccine effectiveness of partial BNT162b2 and mRNA-1273 vaccines for the Delta variant was 53% and 71%, respectively, and that of full vaccination for the Delta variant was 85% and 91%, respectively. In the subgroup analysis, the VE of mRNA-1273 was slightly higher at 7 or more days after full vaccination than at 14 or more days, which we believed was caused by the significantly higher proportion of studies on Delta in the group of 14 or more days than in the group of 7 or more days.
The Delta variant, first identified in India in December 2020 [39,40], quickly became the dominant reported variant in the country in just a few months and is now the most common variant of COVID-19 infections globally, making universal vaccination undoubtedly the best protection against COVID-19 infection in the current situation. The included studies [28,35] showed that partial vaccination had the lowest efficacy against the Beta variants, with BNT162b2 and mRNA-1273 vaccines having efficacies of only 36% and 61%, respectively. However, the number of studies on vaccine efficacy against the Beta variants included in this analysis was too small (less than 3) to provide valuable evidence of vaccine efficacy against Beta as well as Gamma variants.
We also found that mRNA-1273 seemed to be better than BNT162b2 against COVID-19 infection and the VOC after both partial and full vaccination, which is consistent with the results of several other articles [41][42][43]. In particular, when partial vaccination was given, the efficacy of mRNA-1273 against COVID-19 infection (78%) was 17 percentage points higher than that of BNT162b2 (61%). The efficacy of mRNA-1273 against Delta (71%) was 18 percentage points higher than that of BNT162b2 (53%). Differences in VE between the mRNA-1273 and BNT162b2 vaccines might be due to higher mRNA content in mRNA-1273 compared with BNT162b2 and the longer interval between priming and boosting for mRNA-12733 (4 weeks vs. 3 weeks for BNT162b2) [44,45]. Self W.H. also indicated that the VE of mRNA-1273 (VE = 93%) against COVID-19 hospitalization was higher than that of the BNT162b2 vaccine (VE = 88%) (p = 0.011) [46]. We believe that prioritizing the mrNA-1273 vaccine may be a better option in countries and regions where vaccine stocks are insufficient for full vaccination.
There are some limitations in our meta-analysis because TDN studies lack a detailed protocol for consensus, in terms of experimental design. There were six articles involving individuals with COVID-19 symptoms, two articles involving health care workers, and other articles involving individuals or community residents undergoing tests among the 19 articles analyzed, and samples from different sources might have skewed the VE of the vaccines. Each study was not exactly the same in terms of adjustment factors; however, key factors such as age and gender were adjusted for all the literature. In addition, nine articles did not report the strains of COVID-19, and almost all the research was conducted in four countries (US, UK, Canada, and Qatar). These differences may have contributed to the heterogeneity observed in some of the pooled analyses. Due to inconsistent grouping of age, risk factors, and severe disease events in the included studies, we did not extract sufficient data for subgroup analysis of these factors. This analysis also does not analyze the efficacy of long periods after vaccination and protection efficiency against severe illness and death in hospital. Despite the limitations described above, our analyses contribute significantly to the evidence base and highlights the spots that may exist in VE across type of vaccine, status of vaccination, and against VOC.

Conclusions
Irrespective of the BNT162b2 or mRNA-1273 vaccine, the Delta variant significantly weakened vaccine protection after partial vaccination, and full vaccination is highly effective against COVID-19 infection and the VOC. The mRNA-1273 vaccine is more effective against COVID-19 infection and VOC than the BNT162b2 vaccine, both after partial and full vaccination. The evidence in this study provides recommendations for national and regional vaccine policies.

Conflicts of Interest:
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.