Seasonal Influenza Vaccine Effectiveness in Persons Aged 15–64 Years: A Systematic Review and Meta-Analysis

Influenza is a respiratory disease caused by the influenza virus, which is highly transmissible in humans. This paper presents a systematic review and meta-analysis of randomized controlled trials (RCTs) and test-negative designs (TNDs) to assess the vaccine effectiveness (VE) of seasonal influenza vaccines (SIVs) in humans aged 15 to 64 years. An electronic search to identify all relevant studies was performed. The outcome measure of interest was VE on laboratory-confirmed influenza (any strain). Quality assessment was performed using the Cochrane risk-of-bias tool for RCTs and the ROBINS-I tool for TNDs. The search identified a total of 2993 records, but only 123 studies from 73 papers were included in the meta-analysis. Of these studies, 9 were RCTs and 116 were TNDs. The pooled VE was 48% (95% CI: 42–54) for RCTs, 55.4% (95% CI: 43.2–64.9) when there was a match between the vaccine and most prevalent circulating strains and 39.3% (95% CI: 23.5–51.9) otherwise. The TNDs’ adjusted VE was equal to 39.9% (95% CI: 31–48), 45.1 (95% CI: 38.7–50.8) when there was a match and 35.1 (95% CI: 29.0–40.7) otherwise. The match between strains included in the vaccine and strains in circulation is the most important factor in the VE. It increases by more than 25% when there is a match with the most prevalent circulating strains. The laboratorial method for confirmation of influenza is a possible source of bias when estimating VE.


Introduction
Influenza is a respiratory disease resulting from infection with the influenza virus. It is more prevalent during cold periods, with the peak of infections between November and April in the Northern Hemisphere and between June and October in the Southern Hemisphere. The influenza virus is highly transmissible in humans [1]. The World Health Organization (WHO) estimates that there are 1 billion cases of influenza worldwide each year, of which 3-5 million are severe cases [2]. An estimated 650,000 deaths per year result from influenza infection [3]. The most effective way to prevent influenza infection is through vaccination [4]. Seasonal flu vaccination campaigns represent a major investment for countries and governments. It is therefore important to assess the effectiveness of the vaccine.
The two main types of studies used to assess the seasonal influenza vaccine (SIV) performance are randomized controlled trials (RCTs) and observational studies. Among these, the most used are cohort studies and, mainly, the case-control study, known as test-negative design (TND) [5]. RCTs are always conducted for the marketing authorization of the vaccine regardless of the year in question [6]. Vaccine performance is determined by vaccine efficacy (VER), which is equal to VER = (1 − RR) × 100 (1) where RR is the relative risk. These trials are very expensive and time-consuming [7]. Therefore, the application of (1) is not a parsimonious method for real-time monitoring of the efficacy of a SIV. For monitoring the annual effectiveness of different vaccines, TND is used when laboratory confirmation is required [8]. The TND sample is composed of individuals with influenza-like illness (ILI) who access a hospital or other healthcare facilities for a consultation. These individuals are tested for influenza disease, and positive cases are recognized as cases, while negative cases are identified as the controls. The effectiveness of the vaccine is measured by comparing the odds of infection between those vaccinated and unvaccinated [9,10]. The vaccine effect is measured by its effectiveness (VE), which is equal to VE = (1 − OR) × 100 (2) where OR stands for the odds ratio. Several factors can affect the VE and introduce bias in the estimates of the VE. For example, VE can be seriously affected by the mismatch of the virus strains included in the vaccine and those in circulation in each vaccination season. The WHO has developed influenza surveillance and monitoring systems in order to understand which strains are circulating worldwide. Five reference centers located in the US, UK, Australia, Japan and China are responsible for collecting the information issued by each country and pinpointing the strains that are expected to be most prevalent in the following year, and these are the ones that should be in the next SIV [11]. The high rate of viral mutation, which includes fewer marked processes such as antigenic drift and profound changes called antigenic shift, mean that strains are not always as expected, to the notable detriment of the VE. Previous vaccination may bias VE estimation. Natural infection and vaccination interfere with the individual's immune system. On a theoretical level, it is expected that there may be some pre-existing immunity, either from previous infection or vaccination. Thus, resistance to the disease may be favored [12]. However, differences are not always significant [13]. The presence of comorbidities can affect VE. One of the groups at high risk of a severe influenza illness comprehends people with associated health problems. The influence of comorbidities can be analyzed in two ways: the ability of the interaction with the vaccine to be sufficiently robust for good protective capacity and the comorbidities as a determining influence on resistance to infection [14]. The type of substrate used for viral replication, using eggs or cultured cells may affect VE [15]. The time of the vaccination uptake is also relevant [16]. Finally, individual characteristics such as age [14,17], sex [14], conditions such as pregnancy [18] and even mood [19] at the time of the vaccine uptake seem to affect the VE.
Thus, RCTs and TNDs are used in different contexts. RCTs are the gold standard for licensing of use and TNDs are the main tool for monitoring the annual effectiveness of the SIVs [20,21]. However, these test designs apply different measures (VER and VE, respectively) and no relationship has been established between the results observed in RCTs and TNDs. This paper describes a systematic review and meta-analysis of RCTs and TNDs conducted to assess the VER and VE of SIVs in humans aged 15 to 64 years. Despite several papers describing performed reviews of the VER of the SIV or its effectiveness are available [22][23][24][25][26][27], information regarding VE is scarce. In this sense, the main objective of this work is to measure the effect of a vaccine assessed in RCTs and TNDs using a common measure: VE. Other information related to the individuals who participated in each study was also collected to identify possible factors that may influence VE.

Methods
This review adhered to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) criteria [28]. The proposed methodology for the systematic review was registered in the international prospective register of systematic reviews PROSPERO [CRD42023397974].

Search Strategy and Eligibility Criteria
An electronic search was conducted to identify all relevant studies. The literature search was performed in MEDLINE (via PubMed) and Cochrane's library. The search in PubMed was performed using 4.3.0 software [29], package RISmed [30] available at https://www.r-project.org/. The last search date was March 2023. We also screened the reference list of included studies and relevant systematic reviews. The detailed search strategy is available in the register on PROSPERO. Only studies written in English were included to avoid any bias related to a mistranslation.
The included studies verified the following inclusion criteria: -Inclusion of sufficient information to compute the vaccine efficacy/effectiveness; -Articles published between 2013 and 2023; -Influenza was confirmed by a laboratorial method; -Articles that include the human population aged 15 to 65 years; - The participants in TNDs must have received a seasonal influenza vaccine at least 14 days before symptoms onset to be regarded as vaccinated; -Comparator was a placebo (for RCTs) or non-vaccinated (for TND); -Articles published in English.
TND studies were excluded if they reported only pooled data for more than one season.

Interventions
The intervention of interest was vaccination with one of the following seasonal influenza vaccines: trivalent inactivated (TIV), tetravalent inactivated (QIV) or live attenuated (LAIV). Monovalent and noncommercial vaccines were not considered.

Outcome Measure
The outcome of interest was vaccine efficacy or effectiveness on laboratory-confirmed influenza (any strain). PCR (polymerase chain reaction) and rapid virus detection tests were considered as possible methods for the confirmation of infection.
When studies reported both unadjusted and adjusted vaccine effectiveness, the adjusted figure was used in the results as it was considered the less biased estimate of the treatment effect.
Additional outcomes of interest were collected for subgroup analysis.

Data Collection and Analysis
Two authors of this review independently assessed the study eligibility by inspecting the title and abstract. All articles selected from the title/abstract reading were inspected for inclusion with a full-text review by both authors. The information of all selected papers was independently extracted to a form that included study design, participants, sample size, description of intervention, outcomes and quality assessment indicators. Discrepancies in study selection were resolved through consensus.
After the systematic review, a meta-analysis was performed using software's metafor package [31]. When any of the outcome measures were zero, the value of 0.5 was added. The associated confidence intervals are based on the logarithm transformation.
Forest plots were obtained to present a graphical overview (values of VE lower than −200% were omitted).
Between-study heterogeneity was assessed by Cochran's Q and the I 2 statistic. A value higher than 0.75 was regarded as high heterogeneity and the pooled estimate was inter-preted as low-certainty evidence. If the I 2 statistic was lower than 0.25, a fixed effects model was chosen. Otherwise, if the I 2 statistic was higher than 0.25 and lower than 0.75, a random effects model was used. Model estimation was performed using a restricted estimation maximum likelihood methodology with Knapp and Hartung adjustment. The adjusted estimates found in RCTs/TNDs were obtained using logistic regression. Thus, pooling was based on the log RR/OR and standard deviation, with the exponential of the pooled result re-expressed as VER/VE. Funnel plots provided a visual assessment of possible publication bias. The trim and fill method and Egger's test were applied to screen for possible publication bias [32].

Quality Assessment
Two authors independently assessed the included studies for risk of bias using validated critical appraisal tools. Inconsistencies were resolved by a third reviewer.
The Cochrane risk-of-bias tool for randomized trials (RoB 2) was used for RCTs [33]. Data were inputted in the RoB 2 Excel tool to implement them (available on riskofbiasinfo.org website, accessed on 1 March 2023).
TND studies were assessed for risk of bias using the ROBINS-I (Risk of Bias In Nonrandomized Studies of Interventions) tool [34]. Results are presented in tabular form, with the agreed consensus of risk of bias for each of the seven included domains and the overall risk of bias for each study denoted by the highest risk of bias score in any singular domain, as per the ROBINS-I methodology. While unadjusted effectiveness was used where adjusted was not reported, there is clearly a risk of bias associated with the unadjusted estimate.
A sensitivity analysis was produced to control the effect of the high risk of bias studies.

Results
The search identified a total of 2993 records after removing duplicates (see Figure 1). The full text of 172 records was screened for eligibility, 99 of which were excluded. References of excluded studies are reported in Supplementary Table S1. A total of 123 studies from 73 papers were included in the meta-analysis. Of these studies, 9 are RCTs [35][36][37][38][39] and 114 are TNDs . The selection process is detailed in the PRISMA flowchart (see Figure 1).
These articles comprised 86 studies from the Northern Hemisphere and 31 studies from the Southern Hemisphere. Six articles have information from countries in the Northern and in the Southern Hemisphere. In studies performed in more than one European country, the country is referred to as Europe (e.g., I-MOVE studies).
The main characteristics of the RCT studies reported in the articles are summarized in Table 1. Table 2 presents the characteristics of the TND studies.    The RCT sample size ranged from 85 to 7515 in the 9 studies. The reported VER ranged from −2% to 70%. The heterogeneity between studies is low: I 2 = 9.9% and Cochran's Q = 11.5 (p-value 0.18). The TIV was the most-used vaccine (seven studies). QIV and LAIV were used in only one study each. All individuals from one of the studies reported in [35] were HIV positive. Contact with the influenza virus in the study extracted from [39] was deliberately provoked. We find a match between the vaccine strains and the virus in circulation in six studies [36][37][38][39]. In two of the three studies where we verified a mismatch [35,37], it was possible to extract data for the matched strains. The studies extracted from [38] include only pregnant women.
The study by Liebowitz, D. et al. reported positive results for influenza for both ILI and non-ILI patients [39]. As our goal was to compare RCTs to TNDs, the efficacy concerning the influenza-positive illness was reported. The studies extracted from Mcbride, W.J. et al. [37] and Steinhoff, M.C. et al. [38] included two consecutive seasons.
The TND sample size ranged from 62 to 59,150. The TIV was the most-used vaccine (59 studies) followed by QIV (19 studies) in studies that used only 1 type of vaccine. Eighteen studies used both the TIV and QIV vaccines. LAIV was the only choice in three studies. Six studies used all three types of vaccines. The type of vaccine used in eight studies was unclear.
For the RCTs studies, the pooled adjusted VER is equal to 42.8% (95% CI: 30.5-52.9), which is similar to the pooled non-adjusted VER: 42.7% (95% CI: 30.1-53.0). In both models, I 2 = 0 (forest plots not shown). Figure 2 presents a forest plot with the VE computed from the extracted crude values. This allows us to have a common measure to assess a vaccine performance regardless of the type of study (RCT or TND). The VE is represented by individual squares proportional to the precision of the estimates, and the horizontal lines represent the 95% CIs for each included study. The diamond indicates the pooled VE, which is equal to 45.3% (95% CI: 32.1-55.8). For the TND studies, the reported VE ranged from −2% to 70%. The heterogeneity between studies is very high: I 2 = 93.4% and Cochran's Q = 247 (p-value < 0.001).
The pooled VE of the TIV is 48.3% (95% CI: 41.7-54.2) when there is a match between the strains included in the vaccine and the most prevalent in circulation, while it decreases to 40.1% (95% CI: 29.1-49.4) when there is a mismatch.
As for the QIV-type vaccine, studies show an overall effectiveness of 34.3% (95% CI: 29.6-38.7). When both the TIV and QIV vaccines are used in the same study, the effectiveness rate is 37.3% (95% CI: 24.5-47.8). In LAIV studies, the overall effectiveness is only 5.4% (95% CI: −20.7-25.9). Finally, when all three types of vaccines are used within the same study, the effectiveness is equal to 32.8% (95%CI: 12.3-48.5).

Risk of Bias Assessment
The RoB2.0 assessment indicates that one study [33] has a high risk of bias. A summary of the results is presented in Figure 9. In the RCT studies, no evidence of publication bias is found through the funnel plot ( Figure 10). Trim and fill methods and Egger's test (p-value 0.19) do not identify any missing RCT study.  Figure 11 is the funnel plot for the TND studies. We found some evidence of asymmetry, which is confirmed by the trim and fill method (10 missing studies) and Egger's test (p-value < 0.001). However, when restricting to studies where vaccine and circulating strains match, no missing studies were identified by the trim and fill method, and Egger's test p-value increased to 0.04. When restricting to studies where vaccine and circulating strains mismatch, results were similar to the general case. The ROBINS-I assessment tool for intervention was applied to papers involving TND studies. Most of the studies were at a serious/critical risk of bias as the vaccination status was not always based on the individual's records (classification bias). Eleven papers were determined to have a low risk of bias. Fourteen studies failed to provide sufficient information to be classified in at least one of the seven domains analyzed by the tool. The overall results are summarized in Figure 12, and Supplementary Table S4 presents the results by assessment domain.

Subgroup Analysis
A sensitivity analysis was performed by removing the high-bias RCT study [38]. The relative adjusted vaccine efficacy increased by 1.4% and the vaccine effectiveness by less than 0.5% As for the TND studies, those at a critical risk of bias were removed for the sensitivity analysis. The pooled VE showed non-significant variations. However, for the TIV vaccines when there was a mismatch between the circulating and vaccine strains, it decreased by nearly 10%. Table 3 presents the vaccine effectiveness for the studies where the vaccine and circulating strains match and mismatch separately. A 15% increase in vaccine effectiveness was observed when the strains match. When our analysis is restricted to the TIV, the variation is even greater with an increase of more than 20%. Table 3. Comparison of vaccine effectiveness according to the match between circulating and vaccine strains verified in the RCT studies (95% confidence intervals in brackets; a p-value lower than 0.05 is identified with *).

Match
Mismatch p-Value I 2 (%)  Table 4 presents a subgroup analysis for the TND studies. The TIV showed a better performance compared to studies that did not use a TIV (p-value 0.010). A match between the vaccine and circulating strains improves the VE by more than 10% (p-value 0.017). Confirming influenza by PCR results in a higher VE estimate (p-value 0.012).
No significant differences were found between pooled adjusted and non-adjusted VE estimates, studies that include individuals with severe symptoms and studies that included only ILI individuals, studies that included or not only inpatients, and studies performed in different hemispheres.
TIV pooled effectiveness observed in RCT studies is higher than the pooled value obtained from the TND studies when vaccine and circulating strains match, although the difference is not significant (p-value = 0.27, I 2 = 17.4%). When there is a mismatch, the values are similar (around 40% in both cases). Table 4. Comparison of vaccine effectiveness according to several factors observed in TND studies. (95% confidence intervals are presented in brackets; a p-value lower than 0.05 or 0.01 is identified with * or **; a comparison between studies that only use the TIV vaccine and the other; b comparison between studies where vaccine and circulating strains match and mismatch, only for TIV studies; c comparison between studies that only use the QIV vaccine and the other; d comparison between studies that only use the LAIV vaccine and the other, e comparison between studies where vaccine and circulating strains match and mismatch, all studies; f comparison between studies that include individuals with severe symptoms and those that do not; g comparison between studies that include inpatient individuals and those that do not; h comparison between studies that confirm the presence of the influenza virus using only PCR and those that do not).

Discussion
RCTs and TNDs are the most-used study designs to assess the performance of the SIV. Comparing RCT and TND estimates through a common measure (VE) is a relevant subject as the usefulness of TNDs is still discussed in the literature [10]. This explains the focus on individuals aged 15-64 years as they are not, in general, a high-risk group for severe influenza illness. Elderly people were excluded as it would increase the risk of dealing with results extracted from individuals with comorbidities. In addition, RCTs in the elderly population have another vaccine as a comparator [108,109]. Placebo is not used, as expected, because vaccination is recommended [110].
The number of RCT studies found was small. This fact limits the possibility of comparing RCT and TND studies except for TIV vaccines. The VE estimated by RCT studies is 10% higher than the VE estimated through the TNDs, although the difference is not significant. When there is a mismatch, similar values were obtained for both designs. It seems that TND studies are a reliable alternative for the assessment of a vaccine's performance, as it is referred to in [111].
One of the main purposes of a meta-analysis is to compute pooled estimates. However, the pooled VE for the TND is not shown as it would be pointless. The VE of the individual studies varies over a wide range and the measure of the between-studies heterogeneity I 2 is close to 100%. However, it is possible to identify some reasons that explain this high heterogeneity.
From this review, the match between the vaccine and circulating strains arises as the most important factor. In TND studies, a difference of close to 10% was observed. This is in line with what was found in a systematic review of 2016 [112]. In elderly people, even greater differences were reported, between 20 and 30% [113,114]. When comparing TIV and QIV vaccines, we found higher effectiveness values for TIV vaccines. This is true for both RCT and TND studies. This was a surprising result as VE should increase with the number of strains included in the vaccine, although this was already observed in previous work on children [115]. Our understanding is that a match between the strains included in the vaccine and those that are predominantly circulating is the most influential factor. Hence, it is not relevant to have a high number of strains in a vaccine if they do not match the strains the vaccine aims to prevent. This also explains why the RCT study with a sample in which all individuals were HIV positive did not have a low VE, as there was a match. The relevance of the vaccine strains emerges as a key factor in effectiveness. This conclusion is supported by other meta-analyses whose results also point in this direction [116][117][118]. It is also interesting to observe that pooled VE obtained from adjusted and non-adjusted estimates are not significantly different. This leads one to believe that the impact of some of the confounding variables identified in the literature as influencing VE (e.g., prior vaccination) is limited, as some authors have already referred to [119].
PCR tests were used in most studies, although in some cases they were not the exclusive method for the detection of the influenza virus. It was not possible to compare the use of PCR tests with their non-use. Thus, it was only possible to compare the exclusive use of PCR tests with the combined use of more than one type of laboratory test. One of the alternative tests used was the rapid test, which has a lower sensitivity [120]. This lack of sensitivity might be the explanation for a significantly higher VE in the studies that used only PCR tests.

Limitations of the Study
Despite the interesting results found herein, some limitations were evident. LAIV vaccines have very low effectiveness values. The number of studies (3) in which these vaccines are involved is low, so it is not possible to generalize the results. For instance, a VE equal to 44% was found in a systematic review reported in [115].
The TND is validated to assess vaccine effectiveness in outpatients but not in inpatients [57], which limited the possibility to compare disease severity with vaccine effectiveness. This may explain why only in a few studies individuals with acute respiratory infections were found. Thus, it was not possible to assess the impact of symptom severity.
As the number of studies reporting VE by age and strain was very low, it was not possible to assess the effect of the different strains in the VE estimation. Other limitations arise from the high number of TND studies at critical risk of bias and the lack of control over some variables, which could impact the reliability of the results. However, the majority of studies presented VE estimates adjusted for several confounding variables, although these variables were not always the same across the different studies.

Conclusions
This meta-analysis provides important insights into the effectiveness of influenza vaccines, highlighting the crucial role of the match between vaccine strains and those circulating in the population. The findings observed herein provide a basis for future research on the effectiveness of influenza vaccines and suggest that efforts should focus on improving the match between vaccine strains and those circulating in the population.

Supplementary Materials:
The following supporting information can be downloaded at https:// www.mdpi.com/article/10.3390/vaccines11081322/s1, Table S1: references of excluded studies; Table S2: characteristics of the included RCT studies; Table S3: characteristics of the included TND studies; Table S4: ROBINS-I overall assessment of the included TND studies by domain.

Data Availability Statement:
The datasets used and/or analyzed during the current study are available from the corresponding author upon reasonable request.