Effectiveness of the WHO-Authorized COVID-19 Vaccines: A Rapid Review of Global Reports till 30 June 2021

Large clinical trials have proven the efficacy of the COVID-19 vaccine, and the number of studies about the effectiveness rapidly grew in the first half of the year after mass vaccination was administrated globally. This rapid review aims to provide evidence syntheses as a means to complement the current evidence on the vaccine effectiveness (VE) against various outcomes in real-world settings. Databases (PubMed, EMBASE, and MedRxiv) were searched up to 30 June 2021, (PROSPERO ID: 266866). A total of 39 studies were included, covering over 15 million participants from 11 nations. Among the general population being fully vaccinated, the VE against symptomatic SARS-CoV-2 infection was estimated at 89–97%, 92% (95% CI, 78–97%), and 94% (95% CI, 86–97%) for BNT162b2, ChAdOx1, and mRNA-1273, respectively. As for the protective effects against B.1.617.2-related symptomatic infection, the VE was 88% (95% CI, 85.3–90.1%) by BNT162b2 and 67.0% (95% CI, 61.3–71.8%) by ChAdOx1 after full vaccination. This review revealed a consistently high effectiveness of certain vaccines among the general population in real-world settings. However, scarce data on the major variants of SARS-CoV-2 and the shortness of the study time may limit the conclusions to the mRNA vaccines and ChAdOx1.


Introduction
Large randomized control trials have demonstrated the efficacy against SARS-CoV-2 infection to be 70%, 94%, 95%, 78%, and 84% after two doses of the ChAdOx1 (Oxford/AstraZeneca, Oxford, UK), mRNA-1273 (Moderna, Cambridge, MA, USA), BNT162b2 (BioNTech/Pfizer, Mainz, Germany), BBIBP-CorV (Sinopharm, Beijing, China), and Coron-aVac (Sinovac, Beijing, China) vaccines, respectively [1][2][3][4][5]. A single dose of the Ad26.COV2.S (Johnson & Johnson, New Brunswick, NJ, USA) vaccine yielded an efficacy of 82% against severe-critical COVID-19 disease [6]. Based on the good results from clinical trials, COVID-19 vaccination programs have been extensively rolled out in many countries around the world. However, over the first half-year since vaccine administration globally, most countries have full vaccination rates of less than 50% [7]. Billions of people in the world are eagerly waiting for the COVID-19 vaccines on the one hand and questioning how well the vaccines work in the real world on the other hand. As relevant study reports have been released successively, a wide range of vaccine effectiveness (VE) has been noticed. The estimated effectiveness after full vaccination could range from 50% [8] to 100% [9], according to effectiveness studies of the six vaccines (BNT162b2, ChAdOx1, Ad26.COV2.S, mRNA-1273, BBIBP-CorV, CoronaVac), which were listed on the World Health Organization (WHO) Emergency Use Listing as of June 2021 [10].
Given a different efficacy was shown for vaccines developed with different platforms [1][2][3][4], their effectiveness in the real world for different populations needs to be confirmed. A Denmark study showed different VE in different age groups (77% for people ≥ 85 years of age vs. 86% for people ≥ 65 years), and different living/working environments (53% for long-term care facilities dwellers vs. 80% for healthcare workers) [11]. Similarly, the VE may also be different among different countries or races. Attention should also be paid when VE from different studies are compared as VE of a given vaccine depends on what outcomes were chosen in determining VE. A study on the residents of long-term care facilities in Spain showed the VE of mRNA vaccines was 70% against asymptomatic infection and 97% against death [12]. In addition, many SARS-CoV-2 variants have been evolving since the pandemic. B.1.1.7 (alpha), B.1.351 (beta), B.1.617.2 (delta), and P.1 (gamma) variants, which are circulating in the whole world and are causing serious infections and mortality rate that are classified as variants of concern (VOC) by WHO. The VE against these VOC needs to be explored from the literature.
This rapid review aimed to assess the effectiveness of WHO-authorized COVID-19 vaccines, taking into account the aforementioned factors including country, characteristics of the study population, study design, outcomes, and the analysis of the involved VOC.

Materials and Methods
This rapid review was conducted based on the updated guideline of PRISMA 2020 statement and its recommended checklist [13,14]. We registered this review on PROSPERO (ID: 266866) on 13 July 2021. The eligibility criteria are reports evaluating the effectiveness of COVID-19 vaccines in populations aged ≥16 years. Additional inclusion criteria require that the sample size of the vaccinated population should be more than 1000 to have a sufficient event number. Two authors (C.-J.C., Y.S.) performed the literature search in PubMed, EMBASE, and medRxiv. The search terms in PubMed and EMBASE were "effectiveness", "COVID-19 vaccine", and publish time "2021". Preprint articles from medRxiv were searched with the terms "effectiveness COVID-19 vaccine" or "effectiveness SARS-CoV-2 vaccine" in the titles or abstracts.
The article types we reviewed included original investigations, research letters, short communications, and correspondence articles. While screening the title and abstracts of the relevant articles in PubMed, similar research with titles shown on the web page was also checked. We updated our search up to 30 June 2021. C.J.C. and Y.S. contributed to the title and abstract screening for relevance and reviewing of full-text articles against inclusion and exclusion criteria. We excluded the following research: in vitro studies, animal studies, experimental clinical trials, systematic reviews or meta-analyses, diagnostic studies, methodological publications, editorial-style reviews, abstracts of posters, secondary analyses, studies with only immunogenicity data, safety reports or post-infection treatment, and articles with analyses only on a very specific target population, such as veterans, dentists, pregnant women, and patients with malignancy or mental illness.
Ethical approval was not applicable because all the study materials of this review are from the published articles and preprints.
After identification of all relevant articles, quality assessment was performed based on ROBINS-I of Cochrane Handbook to assess the risk of bias. Each bias domain and overall risk of bias was judged as "Low", "Moderate", "Serious", or "Critical" risk of bias based on the check list on the ROBINS-I assessment chart. The extracted data included the following items: author, country, number of vaccinated and unvaccinated participants, study design, age and characteristic of participants, types of vaccine, outcomes, definition about minimal intervals between vaccination (first dose and second dose), and event measurement, involvement of SARS-CoV-2 VOC, and VE with confidence interval (CI) respectively after first dose and second dose of vaccine.
The formula for calculating VE is (1 − hazard ratio for SARS-CoV-2 infection in vaccinated vs. unvaccinated participants) × 100%. In studies that reported the incidence of infection, we calculated the incidence rate ratio (IRR) and converted it to unadjusted VE as (1 − IRR in vaccinated vs. unvaccinated participants) × 100%. The VE from a case-control study was calculated as (1 − odds ratio) × 100%. In case of insufficient data in an article, we contacted the authors to obtain the required information by email. We applied narrative synthesis to process the data from the included studies. As the number of studies of VE constantly grew over our processing period, this review only included reports that were released before 1 July 2021. Distributions of the VE estimates derived from the included studies were further graphically presented by Box plots, according to the study population, brand of vaccine, variant, number of doses, and outcome.

Asymptomatic Infection, Symptomatic Infection, Hospitalization, Critical Disease, and Death
The VE estimates against SARS-CoV-2-related various outcomes are shown in Table 1, Figure 3A-D, and Supplementary Figure S1. Among the general population in the UK, the VE against asymptomatic infection after full vaccination either with BNT162b2 (VE: 58%) or ChAdOx1 (VE: 61%) was not as high as that in Israel (VE: 92%) ( Figure 3A, Table 1A) [33,42]. Much more beneficial effects were found when symptomatic infection reduction was considered as an outcome, with the VE reaching up to 89-97% for BNT162b2 [26,28,33,37,42], 92% for ChAdOx1 [42], and 94% for mRNA-1273 after full vaccination (Table 1, Figure 3B) [26]. Studies in healthcare workers also showed beneficial effects from vaccination (VE: 94-97% by two doses of BNT162b2) [15,16,30], except for a CoronaVac study in Brazil, which showed a low VE against symptomatic infection with the predominant P.1 variant (37%, 95% CI, 54.9-74.2%) even after being fully vaccinated ( Figure 3B, Table 1) [35]. The protective effects become more obvious for more severe outcomes. The VE against critical disease was higher than 90% by BNT162b2 [28,33,40]. A study even demonstrated a VE of 100% for subjects who were fully vaccinated with mRNA-1273 (see Supplementary Figure S1) [40]. Regarding the risk reduction on hospitalization among the general population, the effectiveness was similar among one dose of ChAdOx1 (VE: 88-100%) [31,50], two doses of BNT162b2 (VE: 87-99%) [11,28,33,50], and two doses of mRNA-1273 (VE: 86-100%) ( Figure 3C) [31,40]. Reduction of death for more than 95% was also found by one dose of ChAdOx1 and two doses of mRNA vaccines ( Figure 3D). The VE estimates against COVID-19 hospital admissions after vaccination either with BNT162b2 or ChAdOx1 in older age groups were slightly lower than that of younger age groups [50]. The result of VE across age groups is shown in Supplementary Table S3.

Discussion
This rapid review included 39 studies from 11 countries. Some of these studies may have overlapping populations with data obtained from the same datasets (e.g., national registry) or from the same institution or same region [15][16][17][19][20][21]28,32,34,[36][37][38][39]41,42,45]. By excluding possible double counted participants, this review approximately covered a population of 15 million in total with over 8 million vaccinated subjects. The results of all included studies consistently show that vaccination among the general population in real-world settings, with either BNT162b2, ChAdOx1, or mRNA-1273, substantially reduced the risk of overall SARS-CoV-2 infection. Though the effectiveness of the vaccine in residents of long-term care facilities is lower than that in the general population, studies showed that just one dose of the BNT162b2 or ChAdOx1 vaccine can still provide a VE of 60% or higher. Their protective effectiveness became more obvious as the severity of the clinical endpoint increased. The effectiveness of one dose of ChAdOx1 in preventing hospitalization and death was higher than 80% in the general population and similar to that observed in subjects fully vaccinated with BNT162b2 or mRNA-1273. While the delta variant is becoming the dominant strain circulating in many parts of the world, VE reports on the delta variant are scarce. One recently published study shows that two doses of BNT162b2 can significantly reduce the risk of delta variant-related symptomatic infections [20].
The suboptimal VE observed in the residents of long-term care facilities is probably due to immunosenescence, which leads to defects in innate and adaptive immune responses. Vaccine responses tended to be weaker and declined earlier [51] with a lower titer of antibodies [52][53][54]. Since they are vulnerable to developing SARS-CoV-2 infections and its complications, improved vaccine strategies or further vaccine boosting are required in these vulnerable populations.
It should be noted that VE between vaccines cannot be directly compared across these papers due to the following reasons. The first one is the heterogeneity in the interval between vaccination and the start of event measurement. The VE of the first dose of BNT162b2 against infection was 61% ≥14 days after the vaccination in the study by Pawlowski et al. [40] and 72% ≥21 days after vaccination in the study by Hall et al. [34]. As immunity is gradually building [3], the protective effects would increase over a given period [18,28]. At present, there is no data to suggest the timing of the VE plateau for each vaccine. Second, large variation was noted regarding the dosing intervals. To vaccinate as many people as possible, the UK government decided that the dosing interval for vaccines (whether BNT162b2 or ChAdOx1) could be extended up to 12 weeks [21,37]. In Canada, the administration of the second dose of BNT162b2 and mRNA-1273 was delayed by up to 16 weeks for most individuals due to the disruption of the vaccine supply [26]. There is limited data about the change of the effectiveness for extended dosing intervals. Variations in dosing intervals would be very common in most countries of the world due to the insufficient and unstable vaccine supply. Evidence regarding the effectiveness with appropriate and tolerable dosing intervals are needed [55,56]. Third, some studies [12,29,38,41,44,49] estimated the VE by pooled analyses of two vaccines, in which we cannot obtain the VE of each type of vaccine. Fourth, population bias with unmatched control arms or unequal use of vaccine types across study cohorts could make the data of direct comparison of vaccine types unreliable [20,32,37,40,50]. Different vaccines may target different populations in the roll-out period. For example, old age and healthcare workers are in the priority group of vaccination programs in most countries, but the timescale of supply was different between vaccine types [18,32]. Although the demographic factors were controlled in most studies, residual confounding may not be entirely excluded.
There are several barriers and limitations in this study. First, the literature about the VE against COVID-19 is rapidly evolving and growing exponentially. In this review, 15 of the 39 included studies were preprint articles up to 30 June 2021. While we were processing the review and extracting the data, in 5 July studies were accepted with their data changed in an online version [20,32,35,38,46,57] so that we need to constantly update the data accordingly before submission. Second, these included studies were disproportionately from countries in Europe, North America, and Israel, and were mostly focused on BNT162b2, because these countries had higher priority and access to a large amount of COVID-19 vaccines than most countries in other regions in the first half of 2021 [7]. In addition, shorter dosing intervals and a larger vaccine supply in these nations also make the data of VE associated with BNT162b2 much more comprehensive than other vaccines. Longer dosing intervals made the full vaccination reports of ChAdOx1 much fewer in number than those on mRNA vaccines among the included articles over the past months. Furthermore, the VE assessment reports on CoronaVac and Ad26.COV2.S are also too few to draw conclusions. Third, the VE estimates of all the studies were variable and mostly based on a relatively short study time (range, <2 months to 6 months) with a median follow-up period ranging from 28 to 106 days [15,16,22,33,34,39,40,47]. Further studies are needed to clarify the duration of the protective effects of vaccines. Fourth, studies focused on the VE against the variants of concern, in particularly the delta variant, which evolved as the dominant variant of SARS-CoV-2 in the first half-year of 2021, are scarce [32]. Additionally, the prevalence of the major variants is believed to be quite different among countries and subject to change over time, while the effectiveness of vaccines during the study time might be highly influenced by the local prevalence of the variants [43]. The heterogeneity across studies in the dosing interval, timing of outcome measurement, different target population between vaccine types, variations and shortness of study follow-up period, and scarce data about the prevalence of and the VE against major variants of SARS-CoV-2 all make meta-analyses for studies before 30 June 2021 extremely difficult and may possibly lead to biased results.

Conclusions
This rapid review provided timely and comprehensive evidence on the effectiveness of the ChAdOx1, BNT162b2, and mRNA-1273 vaccines against various COVID-19 infectionrelated endpoints ranging from asymptomatic to critical illness and death. This review highlights the VE across different segments of populations. These real-world results tend to support the implementation of mass vaccination campaigns as public health strategies, and may also help ease skepticism about VE, which is still a common problem in many parts of the world. However, data based on the studies of the first half-year of vaccine administration are not enough and may limit the conclusions regarding the mRNA vaccines and ChAdOx1. Further studies are needed to provide the information on different races/ethnicity, the effects against SARS-CoV-2 variants, and the duration of protection with longer study times.