Next Article in Journal
Clinical and Economic Outcomes Associated with Cell-Based Quadrivalent Influenza Vaccine vs. Standard-Dose Egg-Based Quadrivalent Influenza Vaccines during the 2018–19 Influenza Season in the United States
Previous Article in Journal
Optimising Vaccine Dose in Inoculation against SARS-CoV-2, a Multi-Factor Optimisation Modelling Study to Maximise Vaccine Safety and Efficacy
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Vaccines
Volume 9
Issue 2
10.3390/vaccines9020079
vaccines-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Review

Influenza Vaccine Effectiveness in Mainland China: A Systematic Review and Meta-Analysis

by
Xiaokun Yang
1,
Hongting Zhao
1,
Zhili Li
1,
Aiqin Zhu
2,
Minrui Ren
1,
Mengjie Geng
1,
Yu Li
1,
Ying Qin
1,
Luzhao Feng
3,
Zhibin Peng
1,
Zhijie An
4,
Jiandong Zheng
1,*,
Zhongjie Li
1,* and
Zijian Feng
5,*
1
Division of Infectious Disease, Key Laboratory of Surveillance and Early-Warning on Infectious Disease, Chinese Center for Disease Control and Prevention, Beijing 102206, China
2
Division of Infectious Disease Prevention and Disinfection Management, Shanghai Pudong New Area Center for Disease Control and Prevention, Shanghai 200136, China
3
School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing 100730, China
4
National Immunization Program, Chinese Center for Disease Control and Prevention, Beijing 102206, China
5
Chinese Center for Disease Control and Prevention, Beijing 102206, China
*
Authors to whom correspondence should be addressed.
Vaccines 2021, 9(2), 79; https://doi.org/10.3390/vaccines9020079
Submission received: 18 December 2020 / Revised: 20 January 2021 / Accepted: 20 January 2021 / Published: 23 January 2021
(This article belongs to the Section Influenza Virus Vaccines)
Browse Figures
Versions Notes

Abstract

:
Influenza endangers human health but can be prevented in part by vaccination. Assessing influenza vaccine effectiveness (VE) provides scientific evidence for developing influenza vaccination policy. We conducted a systematic review and meta-analysis of studies that evaluated influenza VE in mainland China. We searched six relevant databases as of 30 August 2019 to identify studies and used Review Manager 5.3 software to analyze the included studies. The Newcastle–Ottawa scale was used to assess the risk of publication bias. We identified 1408 publications, and after removing duplicates and screening full texts, we included 21 studies in the analyses. Studies were conducted in Beijing, Guangzhou, Suzhou, and Zhejiang province from the 2010/11 influenza season through the 2017/18 influenza season. Overall influenza VE for laboratory confirmed influenza was 36% (95% CI: 25–46%). In the subgroup analysis, VE was 45% (95% CI: 18–64%) for children 6–35 months who received one dose of influenza vaccine, and 57% (95% CI: 50–64%) who received two doses. VE was 47% (95% CI: 39–54%) for children 6 months to 8 years, and 18% (95% CI: 0–33%) for adults ≥60 years. For inpatients, VE was 21% (95% CI: −11–44%). We conclude that influenza vaccines that were used in mainland China had a moderate effectiveness, with VE being higher among children than the elderly. Influenza VE should be continuously monitored in mainland China to provide evidence for policy making and improving uptake of the influenza vaccine.

1. Introduction

Influenza is an acute respiratory infectious disease that causes a large burden of disease globally. According to WHO estimates, 5 to 10% of adults and 20 to 30% of children will suffer from seasonal influenza infection every year, resulting in 3 to 5 million cases of severe illness and 290,000–650,000 deaths [1]. In each year of the 2010/2011 through 2014/2015 seasons, there was an estimated 65 to 190 million people infected and 88,100 influenza-associated excess respiratory deaths in China [2]. Although there are antiviral drugs for influenza, such as oseltamivir and zanamivir [3], influenza vaccination is considered the most economical and effective way to prevent influenza [4]. However, with the exception of a few cities where local government subsidize influenza vaccination programs, influenza vaccination has not been introduced in a national, government-funded program for people in mainland China, nor is the influenza vaccine included in health insurance. People pay out-of-pocket for the influenza vaccine, which may contribute to China’s extremely low vaccine uptake rate of 1.5 to 2.2% [5]. Health and economic analyses of seasonal influenza vaccination may help local and national governments justify influenza vaccination programs. An essential component of the health and economic evaluation of vaccines is their effectiveness. Influenza vaccine effectiveness varies year to year. Monitoring influenza VE can provide evidence for program and policy making.
Methods for determining influenza VE vary significantly. Some studies have evaluated the safety and immunogenicity of the influenza vaccine [6,7,8,9] and some influenza VE studies based on influenza-like illness (ILI), which may underestimate influenza VE [10,11,12]. The use of laboratory diagnosed influenza for VE studies has been facilitated by widespread adoption of rapid laboratory testing for influenza virus in medical institutions at all levels of China [13,14,15,16]. We report the results of a systematic review and meta-analysis of influenza VE studies in mainland China to provide evidence for improving VE monitoring and support influenza vaccine policy making.

2. Materials and Methods

2.1. Literature Retrieval

We used the key words “influenza,” “influenza vaccination”, “vaccine effectiveness”, and “China” as search terms to locate published articles. We searched the Wanfang Database, China National Knowledge Infrastructure (CNKI), China Biology Medicine (CBM), and VIP journal database for Chinese language studies; we searched PubMed and Web of Science for English language studies. We used literature reference tracing to identify additional articles and reduce the chance of omitting studies meeting the eligibility criteria.

2.2. Eligibility Criteria

To be eligible for this systematic review and meta-analysis a study had to meet the following criteria: (1) the study setting was in mainland China; (2) the study design was either a test-negative design (TND) case–control study, another type of case–control study, or a cohort study; (3) patients with ILI or acute respiratory infection (ARI) had to have been tested by reverse transcription polymerase chain reaction(RT-PCR) for the influenza virus; (4) the study reported influenza VE; and (5) the study was a post-marketing influenza VE evaluation.

2.3. Data Retrieval

We developed a spreadsheet to organize extracted data. Authors X.Y. and H.Z. extracted data from the included studies; Z.L., M.R., and M.G. checked the data for completeness and accuracy; reviewer disagreements were resolved by Z.L. Information extracted included title, first author, year of publication, study period, study site, study design, study participant description, subject inclusion criteria, samples obtained, and vaccination rate of the influenza positive group and the influenza negative group.

2.4. Literature Quality Evaluation

We used the Newcastle–Ottawa Scale (NOS) [17] to evaluate the quality of the eligible studies from three aspects—population selection, comparability, and exposure evaluation—each aspect receiving up to three points for a total possible score of nine points. We considered scores of more than six points to indicate studies of high quality with low risk of bias; scores of four or five points indicated moderate quality with average risk of bias; scores of three points or fewer indicated low quality with high risk of bias.

2.5. Statistical Analyses

We used Review Manager 5.3 software to conduct meta-analyses. Heterogeneity was assessed by the I2 statistic. The value of I2 reflects the percent of heterogeneity in the total variation; an I2 greater than 50% is evidence of heterogeneity. In our testing for heterogeneity, we used a fixed-effect model (p > 0.05 and I2 < 50%) or a random-effect model (p ≤ 0.05 or I2 ≥ 50%) [18]; we present results as odds ratios (OR) with 95% confidence intervals (CI) and display results in forest plots. We conducted subgroup analyses with delimited data. We compared changes in I2 statistic values when studies with the highest VE, the lowest VE, and the largest sample size were excluded. We considered the results to be stable if the I2 statistic change was <10%. Influenza VE was calculated as (1 − odds ratio) ×100%, where the odds ratio is the odds of vaccination in cases divided by the odds of vaccination in controls.

3. Results

3.1. Basic Information of Eligible Studies

The search terms identified 1408 studies from the six databases—282 from the Wanfang database, 472 from CNKI, 302 from VIP journal database, 282 from CBM, 130 from Web of Science, and 27 from PubMed. Our eligibility review identified 21 articles for including in the analyses [19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39] (Figure 1). The 21 eligible studies were conducted between the 2010–2011 influenza season and the 2017–2018 influenza season. The most common settings were: Beijing; Guangzhou, Guangdong province; Suzhou, Jiangsu province; and Yongkang and Yiwu in Zhejiang province. Study subjects in the Guangzhou studies were children aged 6 months to 8 years old; subjects in Beijing studies included outpatients and hospitalized patients of all ages; subjects in Suzhou studies were 3-to-6-year-old preschool children. Research methods included case–control studies and TND studies. Descriptions of the eligible studies are shown in Table 1.

3.2. Quality Evaluations

Based on Newcastle–Ottawa Scale quality evaluation of the 21 studies (including 26 research results), ten studies received seven points, two studies received six points, and 14 studies received five points. The two most common reasons for score deductions were representativeness of the influenza positive group and absence of a reported response rate. Scores are shown in Table 2.

3.3. Meta Analysis of Influenza VE

Meta-analysis was performed on the 21 included studies (26 separate research results). The heterogeneity test showed that I2 = 84%, p < 0.05, therefore, we used a random-effects model for merging data. Review Manager 5.3 determination showed the overall OR to be 0.64 (95% CI: 0.55–0.75), for a VE of 36% (95% CI: 25–45%). See Figure 2.

3.4. Subgroup Analyses

We conducted subgroup analyses using age of vaccinee, number of doses, and disease severity (hospitalized patients or outpatients). VE for 6 month to 35 month old children who received two doses of influenza vaccine was 57% (95% CI: 50–64%); overall VE for children 6 months to 8 years of age was 47% (95% CI: 39–54%); VE for people over 60 years old was the lowest, and VE for hospitalized patients was 21% (95% CI: −11–44%). Detailed results are shown in Table 3, and meta-analysis results are shown in Figure 3, Figure 4, Figure 5, Figure 6, Figure 7 and Figure 8.

3.5. Sensitivity Analysis and Publication Bias

The highest estimated influenza VE reported in the included studies was 66% (95% CI: 41–81%) by Wang Yin and colleagues [37]. We excluded this study and re-conducted the meta-analysis, finding an OR of 0.65 (95% CI: 0.56–0.76), for a VE of 35% (95% CI: 23–44%). Ma Chunna and colleagues reported the lowest influenza VE [29], which was −19% (95% CI: −47–4%), and had the study with the largest sample size. After excluding Ma’s study, the recalculated OR was 0.62 (95% CI: 0.54–0.72), for a VE of 38% (95% CI: 28–46%). The difference in ORs before and after excluding the studies was less than 10%, implying stability of the meta-analyses. Figure 9 shows a scatter plot (funnel plot) that plots OR on the horizontal axis and the standard error of log(OR) on the vertical axis. The distribution of scattered points was approximately symmetric, indicating that risk of publication bias was relatively low.

4. Discussion

Influenza vaccination is widely considered to be the most effective and cost-beneficial way to prevent influenza [4]. Evaluating influenza VE is very important for supporting policy making. We conducted a meta-analysis of 21 published studies set in mainland China and found that the overall VE was 36% during the 2010/2011 through 2017/2018 influenza seasons, showing a moderately protective effect of vaccination. Influenza VE was higher (57%) among young children 6 to 35 months of age who received two doses influenza vaccine for their first time receiving influenza vaccine; VE was 45% among same-age children who received one dose of influenza vaccine. We found that influenza VE was higher among children than among individuals over 60 years of age.
Two studies from the United States found that during the 2003–2005 season and the 2005–2007 season, VE for 6 to 59 month children who received two doses of vaccine was 57% (95% CI: 28–74%) and 56% (95% CI: 25–74%) [40,41], which is consistent with our findings. However, among children in the two U.S. studies who received a single dose, the vaccine showed no significant protective effect. This is in contrast to our meta-analysis that included six studies and found a protective effect with one dose—although lower than for two doses. This may be due in part to a larger sample size in our meta-analysis that could provide more statistical power.
A retrospective study in Italy [42] found that influenza VE in children was 37.1% (95% CI: 22.2–49.2%) during the 2010/2011 to 2017/2018 influenza seasons. Our findings of higher VE may be because we did not adjust for age, gender, and epidemic season in our meta-analyses, while the study conducted in Italy did. Both studies showed better protection for children from 2010/2011 through 2017/2018 seasons than for elderly adults. Using a fixed-effect model, we found that for people over 60 years of age, VE was only 18%. Such a low protective effect is unsatisfactory. Lower VE among the elderly may be due to immunosenescence. Studies have shown that vaccine-mediated antibody titers among the elderly are lower than those among younger individuals [43,44,45,46]. Different influenza vaccination strategies for the elderly are needed to overcome immunosenescence. The US Food and Drug Administration (FDA) approved a high-dose trivalent inactivated influenza vaccine in 2009, increasing each antigen component from the standard dose of 15 μg to 60 μg [47]. High-dose influenza vaccines demonstrate better efficacy against laboratory-confirmed influenza [48,49,50,51] and better effectiveness against confirmed influenza, influenza-related medical visits, hospitalization, and death [52,53,54]. The findings from our study support the need for promoting development and introduction of high-dose influenza vaccines and adjuvant vaccines that are more suitable for the elderly in mainland China.
A 2016 review found that influenza VE, assessed with test-negative design studies in inpatient settings, had similar results to TND assessments in outpatient settings [55]. A study set in Spain found that influenza VE was 34% (95% CI: 6–54%) among outpatients and 32% (95% CI: 15–45%) among hospitalized patients during the 2010/2011 to 2015/2016 seasons [56]. A study from the U.S. among adults over 18 years of age conducted from 2015 to 2018 [57] showed that influenza VE was 31% (95% CI: 26–37%) for outpatients and 36% (95% CI: 27–44%) for inpatients. We used a random-effect model to evaluate VE for outpatients and inpatients and found that VE was 21% (95% CI: −11–44%) for inpatients and 13% (95% CI: −10–32%) for outpatients—lower than the findings from these other studies. It is possible that vaccine-circulating-strain mismatch is partially responsible. For example, for the 2014/2015 influenza season, the match was poor, and Ma Chunna and colleagues [29] found that influenza vaccination had no protective effect among outpatients. Zhang Yi and colleagues [30] also found no protective effect among hospitalized patients of all age groups in Beijing in the 2015/2016 influenza season. The reason for the poor VE, could also be related to the extremely low influenza vaccination rate in mainland China. Low coverage makes VE assessment more difficult, since VE study sample sizes need to be larger when vaccination rates are low. Evaluation of influenza VE will be facilitated by conducting studies in cities with higher vaccination coverage levels, which currently tend to be cities in which local governments sponsor seasonal influenza vaccination among targeted groups, such as school children or the elderly.
Our study has several limitations. First, the studies we included made some statistical adjustments in their OR calculations, but the adjusting factors were not the same in all studies. Therefore, we did not use adjustment factors in Revman 5.3 software for our meta-analysis. Second, our study only involved only two age groups—6 months to 8 years and over 60 years—limiting our ability to conduct additional subgroup analyses. Third, we did not analyze VE by influenza virus types, primarily due to lack of information about virus types in the studies. Using a standardized protocol in the future, we should be able to assess the VE of different ages and different virus types. Fourth, too few studies were based on laboratory confirmed cases of influenza in mainland China—only Beijing, Suzhou, Zhejiang and Guangzhou regularly used laboratory confirmation. The settings for our meta-analysis were in northern and southern China, which misses some variation in the epidemic seasons in mainland China. Establishing a wider area for VE studies will be useful in the future. Finally, we did not analyze VE by influenza season due to an insufficient number of VE studies.
There are many challenges and opportunities for comprehensive evaluation of influenza VE in mainland China [58]. Vaccine-circulating-strain mismatch, vaccine type and technology, virus strains, vaccination season, repeated vaccination, northern vs. southern influenza seasons, among other factors can be used to understand the many influences of influenza VE. It is essential to evaluate influenza VE across a much larger scale and with systematic and strategic use of a unified research protocol.

5. Conclusions

Based on published studies set in mainland China that used laboratory-confirmed influenza to evaluate influenza VE, we found that influenza vaccination has a moderate protective effect against medically-attended influenza, and that influenza VE among children is higher than among the elderly. Influenza VE evaluations should be conducted continuously, and with a unified research protocol to provide a scientific basis for formulation and adjustment evidence-based influenza vaccination policy.

Author Contributions

Conceptualization, X.Y., H.Z. and Z.L. (Zhongjie Li); methodology, X.Y., Z.L. (Zhongjie Li) and Z.F.; software, X.Y., H.Z., and Z.L. (Zhili Li); validation, X.Y., H.Z., M.R., Z.L. (Zhili Li), A.Z., and M.G.; formal analysis, X.Y., H.Z., Z.L. (Zhili Li), and Z.L. (Zhongjie Li); data curation, X.Y., and H.Z.; writing—original draft preparation, X.Y. and Z.L. (Zhongjie Li); writing—review and editing, M.G., Y.L., Y.Q., Z.P., L.F., Z.A., J.Z., Z.L. (Zhongjie Li), Z.F.; visualization, X.Y. and A.Z.; funding acquisition, Z.P., L.F., Z.L. and J.Z. All authors have read and agreed to the published version of the manuscript.

Funding

This study was supported by the National Science and Technology Major Project (No. 2018ZX10713001-005), the National Natural Science Foundation of China (91846302), the China-US Collaborative Program on Emerging and Re-emerging Infectious Diseases, and Emergency Response Mechanism Operation Program, Chinese Center for Disease for Disease Control and Prevention (No.131031001000015001). The founders had no role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Acknowledgments

We authors would like to thank Suizan Zhou for sharing the most recent influenza vaccination data in Guangzhou. We appreciate the English editing of the manuscript by Lance Rodewald, -the National Immunization Program of China CDC.

Conflicts of Interest

The authors of this study declare that they do not have any potential conflicts of interest.

References

  1. World Health Organization. Influenza (Seasonal). Available online: https://www.who.int/news-room/fact-sheets/detail/influenza-(seasonal) (accessed on 10 September 2019).
  2. Li, L.; Liu, Y.N.; Wu, P.; Peng, Z.B.; Wang, X.L.; Chen, T.; Wong, J.Y.T.; Yang, J.; Bond, H.S.; Wang, L.J.; et al. Influenza-associated excess respiratory mortality in China, 2010–15: A population-based study. Lancet Public Health 2019, 4, e473–e481. [Google Scholar] [CrossRef] [Green Version]
  3. Wang, Y.M.; Cao, B. Perspectives of antiviral drugs used on influenza. Zhonghua Liu Xing Bing Xue Za Zhi 2018, 39, 1051–1059. [Google Scholar] [PubMed]
  4. Paules, C.; Subbarao, K. Influenza. Lancet 2017, 390, 697–708. [Google Scholar] [CrossRef]
  5. Yang, J.; Atkins, K.E.; Feng, L.Z.; Pang, M.F.; Zheng, Y.M.; Liu, X.X.; Cowling, B.J.; Yu, H.J. Seasonal influenza vaccination in China: Landscape of diverse regional reimbursement policy, and budget impact analysis. Vaccine 2016, 34, 5724–5735. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  6. Zhang, P.R.; Zhu, X.P.; Zhou, L.J.; Liu, Y.Q.; Fan, Y.; Chen, G.; Chen, Z.; Liu, Y.; Sun, H.Y.; Wu, J.L. Safety and immunological effect of domestic split influenza virus vaccine. Zhonghua Yu Fang Yi Xue Za Zhi 2009, 43, 615–618. [Google Scholar] [PubMed]
  7. Duan, W.; Yang, P.; Shi, W.X.; Peng, X.M.; Liang, H.J.; Wang, Q.Y. Research on safety and cost-benefit of influenza vaccine. Guo Ji Bing Du Xue Za Zhi 2014, 21, 241–244. [Google Scholar]
  8. Wang, R.Q.; Tang, Y.Q.; Liu, Z.C. Safety and immunogenicity evaluation of domestic influenza split vaccine. Zhongguo Wei Sheng Jian Yan Za Zhi 2008, 18, 340–342. [Google Scholar]
  9. Wang, P.; Zhang, X.W.; Song, Y.F.; Yin, H.B.; Liu, L.J.; Che, L.; Li, H.; Liu, Y.; Chen, J.T. Safety and immunogenicity on the formulation of trivalent split influenza vaccine among healthy people aged over 18 years. Zhonghua Liu Xing Bing Xue Za Zhi 2011, 32, 120–124. [Google Scholar]
  10. Liu, M.; Liu, G.F.; Wang, Y.; Zhao, W.; Wang, L.; Shi, W.; Wen, S.Y. Study on the effectiveness and cost benefit of influenza vaccine on elderly population in Beijing city. Zhonghua Liu Xing Bing Xue Za Zhi 2005, 26, 412–416. [Google Scholar]
  11. Wang, J.Z. Observation on the preventive effect of influenza vaccine in children. Lin Chuang Yi Xue Yan Jiu Yu Shi Jian 2016, 1, 98. [Google Scholar]
  12. Bian, G.L.; Xu, G.Z.; Zhu, L.L. Study on the Efficacy of Infuenza Vaccine among Different populations. Zhejiang Yu Fang Yi Xue 2010, 22, 19. [Google Scholar]
  13. Merckx, J.; Wali, R.; Schiller, I.; Caya, C.; Gore, G.C.; Chartrand, C.; Dendukuri, N.; Papenburg, J. Diagnostic Accuracy of Novel and Traditional Rapid Tests for Influenza Infection Compared With Reverse Transcriptase Polymerase Chain Reaction: A Systematic Review and Meta-analysis. Ann. Intern. Med. 2017, 167, 394–409. [Google Scholar] [CrossRef] [PubMed]
  14. Huang, H.; Chen, Q.; Liang, H.M.; Zhou, T.X.; Zhang, X.B.; Peng, C.J. The clinical value of rapid diagnosis of influenza virus. Int. J. Lab. Med. 2012, 33, 1932–1933. [Google Scholar]
  15. Chen, C.; Fan, Q.Q.; Chen, G.; Zhu, M.Q.; Wu, J.; Zhang, W.H.; Jin, J.L. A new silver amplification immunochromatography system compared with conventional rapid antigen assay for the diagnosis of influenza A and B. Zhongguo Gan Ran Yu Hua Liao Za Zhi 2017, 17, 29–32. [Google Scholar]
  16. Zhang, X.; Yan, H.P.; Ma, Y.X.; Wang, Y.; Zhao, Y.; Zhang, H.P. Evaluation of nucleic acid amplification assay and rapid antigen assay of nasopharynx swabs and oropharynx swabs from fu-like patients in diagnosis of flu A. Zhonghua Chuan Ran Bing Za Zhi 2011, 29, 154–157. [Google Scholar]
  17. Wells, G.A.; Shea, B.; O’Connell, D.; Peterson, J.; Welch, V.; Losos, M.; Tugwell, P. The Newcastle-Ottawa Scale (NOS) for Assessing the Quality of Non-Randomised Studies in Meta-Analyses. Available online: http://www.ohri.ca/programs/clinical_epidemiology/oxford.asp (accessed on 10 September 2019).
  18. McKenzie, J.E.; Beller, E.M.; Forbes, A.B. Introduction to systematic reviews and meta-analysis. Respirology 2016, 21, 626–637. [Google Scholar] [PubMed] [Green Version]
  19. Fu, C.X.; He, Q.; Li, Z.T.; Xu, J.X.; Hu, W.H.; Dong, Z.Q.; Liu, X.X.; Zhang, C.H.; Zhang, D.F.; Nie, J.; et al. A case-cohort study of the protection of the children against influenza after the licensure of influenza vaccine in 2010–2011. Re Dai Yi Xue Za Zhi 2013, 13, 301–303. [Google Scholar]
  20. Fu, C.X.; He, Q.; Li, Z.T.; Xu, J.X.; Li, Y.Q.; Lu, J.Y.; Li, K.B.; Yang, Q.Y.; Dong, Z.Q.; Liu, X.Y.; et al. Seasonal influenza vaccine effectiveness among children, 2010–2012. Influenza Other Respir. Viruses 2013, 7, 1168–1174. [Google Scholar] [CrossRef] [Green Version]
  21. He, Q.; Li, Z.T.; Lu, J.Y.; Li, K.B.; Xu, J.X.; Hu, W.H.; Dong, Z.Q.; Zhang, C.H.; Wang, M.; Zhang, D.F.; et al. A case-control study of seasonal influenza vaccine effectiveness for A(H1N1)pdm09 among children. Re Dai Yi Xue Za Zhi 2014, 14, 599–602. [Google Scholar]
  22. Fu, C.X.; Xu, J.X.; Lin, J.Y.; Wang, M.; Li, K.B.; Ge, J.; Thompson, M.G. Concurrent and cross-season protection of inactivated influenza vaccine against A(H1N1)pdm09 illness among young children: 2012–2013 case-control evaluation of influenza vaccine effectiveness. Vaccine 2015, 33, 2917–2921. [Google Scholar] [CrossRef]
  23. He, Q.; Xu, J.Q.; Wang, M.; Shen, J.C.; Zhang, C.H.; Fu, C.X. A case-control study of effectiveness of seasonal influenza vaccine in children in 2013–2014. Zhong Hua Ji Bing Kong Zhi Za Zhi 2016, 20, 1018–1021. [Google Scholar]
  24. Fu, C.X.; Greene, C.M.; He, Q.; Liao, Y.; Wan, Y.M.; Shen, J.C.; Rong, C.; Zhou, S.Z. Dose effect of influenza vaccine on protection against laboratory-confirmed influenza illness among children aged 6 months to 8 years of age in southern China, 2013/14–2015/16 seasons: A matched case-control study. Hum. Vaccin Immunother. 2020, 16, 595–601. [Google Scholar] [CrossRef] [PubMed]
  25. Yang, P.; Thompson, M.G.; Ma, C.N.; Shi, W.X.; Wu, S.S.; Zhang, D.T.; Wang, Q.Y. Influenza vaccine effectiveness against medically-attended influenza illness during the 2012–2013 season in Beijing, China. Vaccine 2014, 32, 5285–5289. [Google Scholar] [CrossRef]
  26. Zhang, L.; Pan, Y.; Ma, C.N.; Duan, W.; Sun, Y.; Wu, S.S.; Zhang, M.; Tian, Y.; Zheng, Y.; Yang, P.; et al. Moderate influenza vaccine effectiveness against influenza A(H1N1)pdm09 virus and low effectiveness against A(H3N2) virus among older adults during 2013–2014 influenza season in Beijing, China. Hum. Vaccin Immunother. 2018, 14, 1323–1330. [Google Scholar] [CrossRef] [PubMed]
  27. Qin, Y.; Zhang, Y.; Wu, P.; Feng, S.; Zheng, J.D.; Yang, P.; Pan, Y.; Wang, Q.Y.; Feng, L.Z.; Pang, X.H.; et al. Influenza vaccine effectiveness in preventing hospitalization among Beijing residents in China, 2013–2015. Vaccine 2016, 34, 2329–2333. [Google Scholar] [CrossRef] [Green Version]
  28. Zhang, L.; Yang, P.; Thompson, M.G.; Pan, Y.; Ma, C.N.; Wu, S.S.; Sun, Y.; Zhang, M.; Duan, W.; Wang, Q.Y. Influenza Vaccine Effectiveness in Preventing Influenza Illness among Children during School-Based Outbreaks in the 2014–2015 Season in Beijing, China. Pediatr. Infect. Dis. J. 2017, 36, e69–e75. [Google Scholar] [CrossRef] [PubMed]
  29. Ma, C.N.; Pan, Y.; Zhang, L.; Zhang, Y.; Wu, S.S.; Sun, Y.; Duan, W.; Zhang, M.; Wang, Q.Y.; Yang, P. Influenza vaccine effectiveness against medically attended influenza illness in Beijing, China, 2014/15 season. Hum. Vaccin Immunother. 2017, 13, 2379–2384. [Google Scholar] [CrossRef]
  30. Zhang, Y.; Wu, P.; Feng, L.Z.; Yang, P.; Pan, Y.; Feng, S.; Qin, Y.; Zheng, J.D.; Puig-Barberà, J.; Muscatello, D. Influenza vaccine effectiveness against influenza-associated hospitalization in 2015/16 season, Beijing, China. Vaccine 2017, 35, 3129–3134. [Google Scholar] [CrossRef] [PubMed]
  31. Zhang, L.; Pan, Y.; Hackert, V.; van der Hoek, W.; Meijer, A.; Krafft, T.; Yang, P.; Wang, Q.Y. The 2015–2016 influenza epidemic in Beijing, China: Unlike elsewhere, circulation of influenza A(H3N2) with moderate vaccine effectiveness. Vaccine 2018, 36, 4993–5001. [Google Scholar] [CrossRef]
  32. Wu, S.S.; Pan, Y.; Zhang, X.X.; Zhang, L.; Duan, W.; Ma, C.N.; Zhang, Y.; Zhang, M.; Sun, Y.; Yang, P.; et al. Influenza vaccine effectiveness in preventing laboratory-confirmed influenza in outpatient settings: A test-negative case-control study in Beijing, China, 2016/17 season. Vaccine 2018, 36, 5774–5780. [Google Scholar] [CrossRef]
  33. Zhang, D.T.; Chu, Y.H.; Li, H.J.; Zhao, X.J.; Liu, Z.C.; Zhou, L.; Song, X.X.; Chen, Y.L.; Han, J.T.; Liang, J.B.; et al. Effectiveness of seasonal infuenza vaccine against severe acute respiratory infections. Guo Ji Bing Du Xue Za Zhi 2019, 26, 77–81. [Google Scholar]
  34. Zhang, L.; van der Hoek, W.; Krafft, T.; Pilot, E.; Asten, L.V.; Lin, G.; Wu, S.; Duan, W.; Yang, P.; Wang, Q. Influenza vaccine effectiveness estimates against influenza A(H3N2) and A(H1N1)pdm09 among children during school-based outbreaks in the 2016–2017 season in Beijing, China. Hum. Vaccin Immunother. 2020, 16, 816–822. [Google Scholar] [CrossRef]
  35. Zhang, D.T.; Zhang, Y.; Wang, Q.Y.; Lock, J.; Pan, Y.; Cui, S.; Yang, P.; Hu, Y. The effectiveness of influenza vaccination in preventing hospitalizations in elderly in Beijing, 2016–18. Vaccine 2019, 37, 1853–1858. [Google Scholar] [CrossRef] [PubMed]
  36. Luo, S.Y.; Zhu, J.L.; Lv, M.Z.; Hu, Y.Q.; Cheng, H.; Zhang, G.M.; Chen, G.S.; Wu, X.H. Evaluation of the influenza vaccine effectiveness among children aged 6 to 72 months based on the test-negative case control study design. Zhong Hua Yu Fang Yi Xue Za Zhi 2019, 53, 576–580. [Google Scholar]
  37. Wang, Y.; Zhang, T.; Chen, L.L.; Greene, C.; Ding, Y.F.; Cheng, Y.; Yang, C.; Zeng, S.S.; Hua, J.; Zhou, S.Z. Seasonal influenza vaccine effectiveness against medically attended influenza illness among children aged 6–59 months, October 2011–September 2012: A matched test-negative case-control study in Suzhou, China. Vaccine 2016, 34, 2460–2465. [Google Scholar] [CrossRef] [PubMed]
  38. Wang, Y.; Chen, L.L.; Cheng, Y.J.; Zhou, S.Z.; Pang, Y.Y.; Zhang, J.; Greene, C.M.; Song, Y.; Zhang, T.; Zhao, G.M. Potential impact of B lineage mismatch on trivalent influenza vaccine effectiveness during the 2015–2016 influenza season among nursery school children in Suzhou, China. Hum. Vaccin Immunother. 2018, 14, 630–636. [Google Scholar] [CrossRef] [PubMed]
  39. Wang, Y.; Chen, L.L.; Yu, J.; Pang, Y.Y.; Zhang, J.; Zhang, T.; Zhao, G.M. The effectiveness of influenza vaccination among nursery school children in China during the 2016/17 influenza season. Vaccine 2018, 36, 2456–2461. [Google Scholar] [CrossRef]
  40. Eisenberg, K.W.; Szilagyi, P.G.; Fairbrother, G.; Griffin, M.R.; Staat, M.; Shone, L.P.; Weinberg, G.A.; Hall, C.B.; Poehling, K.A.; Edwards, K.M.; et al. Vaccine effectiveness against laboratory-confirmed influenza in children 6 to 59 months of age during the 2003–2004 and 2004–2005 influenza seasons. Pediatrics 2008, 122, 911–919. [Google Scholar] [CrossRef] [Green Version]
  41. Staat, M.A.; Griffin, M.R.; Donauer, S.; Edwards, K.M.; Szilagyi, P.G.; Weinberg, G.A.; Hall, C.B.; Prill, M.M.; Chaves, S.S.; Bridges, C.B.; et al. Vaccine effectiveness for laboratory-confirmed influenza in children 6–59 months of age, 2005–2007. Vaccine 2011, 29, 9005–9011. [Google Scholar] [CrossRef]
  42. Colucci, M.E.; Affanni, P.; Cantarelli, A.; Caruso, L.; Bracchi, M.T.; Capobianco, E.; Zoni, R.; Paini, G.; Odone, A.; Mohieldin, M.I.M.M.; et al. Influenza vaccine effectiveness in children: A retrospective study on eight post-pandemic seasons with trivalent inactivated vaccine. Acta Biomed. 2020, 91, 63–70. [Google Scholar]
  43. Derhovanessian, E.; Pawelec, G. Vaccination in the elderly. Microb. Biotechnol. 2012, 5, 226–232. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  44. Parodi, V.; de Florentiis, D.; Martini, M.; Filippo, A. Inactivated Influenza Vaccines. Drugs Aging 2011, 28, 93–106. [Google Scholar] [CrossRef] [PubMed]
  45. Smetana, J.; Chlibek, R.; Shaw, J.; Splino, J.; Prymula, R. Influenza vaccination in the elderly. Hum. Vaccines Immunother. 2017, 14, 540–549. [Google Scholar] [CrossRef] [PubMed]
  46. Goodwin, K.; Viboud, C.; Simonsen, L. Antibody response to influenza vaccination in the elderly: A quantitative review. Vaccine 2006, 24, 1159–1169. [Google Scholar] [CrossRef] [PubMed]
  47. Centers for Disease Control and Prevention (CDC). Interim results: State-specific seasonal influenza vaccination coverage—United States, August 2009–January 2010. MMWR Morb. Mortal. Wkly. Rep. 2010, 59, 477–484. [Google Scholar]
  48. Wilkinson, K.; Wei, Y.C.; Szwajcer, A.; Rabbani, R.; Zarychanski, R.; Abou-Setta, A.M.; Mahmud, S.M. Efficacy and safety of high-dose influenza vaccine in elderly adults: A systematic review and meta-analysis. Vaccine 2017, 35, 2775–2780. [Google Scholar] [CrossRef]
  49. Falsey, A.R.; Treanor, J.J.; Tornieporth, N.; Capellan, J.; Gorse, G.J. Randomized, double-blind controlled phase 3 trial comparing the immunogenicity of high-dose and standard-dose influenza vaccine in adults 65 years of age and older. J. Infect. Dis. 2009, 200, 172–180. [Google Scholar] [CrossRef] [Green Version]
  50. Couch, R.B.; Winokur, P.; Brady, R.; Belshe, R.; Chen, W.H.; Cate, T.R.; Sigurdardottir, B.; Hoeper, A.; Graham, I.L.; Edelman, R.; et al. Safety and immunogenicity of a high dosage trivalent influenza vaccine among elderly subjects. Vaccine 2007, 25, 7656–7663. [Google Scholar] [CrossRef] [Green Version]
  51. DiazGranados, C.A.; Dunning, A.J.; Kimmel, M.; Kirby, D.; Treanor, J.; Collins, A.; Pollak, R.; Christoff, J.; Earl, J.; Landolfi, V.; et al. Efficacy of high-dose versus standard-dose influenza vaccine in older adults. N. Engl. J. Med. 2014, 371, 635–645. [Google Scholar] [CrossRef]
  52. Shay, D.K.; Chillarige, Y.; Kelman, J.; Forshee, R.A.; Foppa, I.M.; Wernecke, M.; Lu, Y.; Ferdinands, J.M.; Iyengar, A.; Fry, A.M.; et al. Comparative Effectiveness of High-Dose Versus Standard-Dose Influenza Vaccines among US Medicare Beneficiaries in Preventing Post-Influenza Deaths During 2012–2013 and 2013–2014. J. Infect. Dis. 2017, 215, 510–517. [Google Scholar] [CrossRef] [Green Version]
  53. Chit, A.; Becker, D.L.; DiazGranados, C.A.; Maschio, M.; Yau, E.; Drummond, M. Cost-effectiveness of high-dose versus standard-dose inactivated influenza vaccine in adults aged 65 years and older: An economic evaluation of data from a randomised controlled trial. Lancet Infect. Dis. 2015, 15, 1459–1466. [Google Scholar] [CrossRef]
  54. Izurieta, H.S.; Thadani, N.; Shay, D.K.; Lu, Y.; Maurer, A.; Foppa, I.M.; Franks, R.; Pratt, D.; Forshee, R.A.; MaCurdy, T.; et al. Comparative effectiveness of high-dose versus standard-dose influenza vaccines in US residents aged 65 years and older from 2012 to 2013 using Medicare data: A retrospective cohort analysis. Lancet Infect. Dis. 2015, 15, 293–300. [Google Scholar] [CrossRef] [Green Version]
  55. Feng, S.; Cowling, B.J.; Sullivan, S.G. Influenza vaccine effectiveness by test-negative design—Comparison of inpatient and outpatient settings. Vaccine 2016, 34, 1672–1679. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  56. Castilla, J.; Martínez-Baz, I.; Navascués, A.; Casado, I.; Aguinaga, A.; Díaz-González, J.; Delfrade, J.; Guevara, M.; Ezpeleta, C. Comparison of influenza vaccine effectiveness in preventing outpatient and inpatient influenza cases in older adults, northern Spain, 2010/11 to 2015/16. Euro Surveill. 2018, 23, 16-00780. [Google Scholar] [CrossRef]
  57. Tenforde, M.W.; Chung, J.; Smith, E.R.; Talbot, H.K.; Trabue, C.H.; Zimmerman, R.K.; Silveira, F.P.; Gaglani, M.; Murthy, K.; Monto, A.S.; et al. Influenza vaccine effectiveness in inpatient and outpatient settings in the United States, 2015–2018. Clin. Infect. Dis. 2020, ciaa407. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  58. Ainslie, K.E.C.; Haber, M.; Orenstein, W.A. Challenges in estimating influenza vaccine effectiveness. Expert Rev. Vaccines 2019, 18, 615–628. [Google Scholar] [CrossRef] [PubMed]
Vaccines 09 00079 g001 550
Figure 1. Literature screening process for systematic reviews and meta-analysis.
Figure 1. Literature screening process for systematic reviews and meta-analysis.
Vaccines 09 00079 g001
Vaccines 09 00079 g002 550
Figure 2. Meta-analysis of influenza vaccine effectiveness (VE) in mainland China.
Figure 2. Meta-analysis of influenza vaccine effectiveness (VE) in mainland China.
Vaccines 09 00079 g002
Vaccines 09 00079 g003 550
Figure 3. Subgroup analysis of children 6–35 months who received one dose of influenza vaccine.
Figure 3. Subgroup analysis of children 6–35 months who received one dose of influenza vaccine.
Vaccines 09 00079 g003
Vaccines 09 00079 g004 550
Figure 4. Subgroup analysis of children 6–35 months who received two doses of influenza vaccine.
Figure 4. Subgroup analysis of children 6–35 months who received two doses of influenza vaccine.
Vaccines 09 00079 g004
Vaccines 09 00079 g005 550
Figure 5. Subgroup analysis of children aged 6 months to 8 years old.
Figure 5. Subgroup analysis of children aged 6 months to 8 years old.
Vaccines 09 00079 g005
Vaccines 09 00079 g006 550
Figure 6. Subgroup analysis of adults more than 60 years old.
Figure 6. Subgroup analysis of adults more than 60 years old.
Vaccines 09 00079 g006
Vaccines 09 00079 g007 550
Figure 7. Subgroup analysis of inpatients.
Figure 7. Subgroup analysis of inpatients.
Vaccines 09 00079 g007
Vaccines 09 00079 g008 550
Figure 8. Subgroup analysis of outpatients.
Figure 8. Subgroup analysis of outpatients.
Vaccines 09 00079 g008
Vaccines 09 00079 g009 550
Figure 9. Funnel plot for meta-analysis of influenza VE.
Figure 9. Funnel plot for meta-analysis of influenza VE.
Vaccines 09 00079 g009
Table
Table 1. Characteristics of 21 studies included in the meta-analysis.
Table 1. Characteristics of 21 studies included in the meta-analysis.
(Ref.)Published YearStudy PeriodStudy PlaceStudy ObjectsStudy MethodVaccination Rate
in Case Group		Vaccination Rate
in Control Group
[19]20132010/09–2011/09Guang Zhou6 months~5 years community children case–control study16.9% (52/308)36.0% (279/774)
[20]20132011/01–2011/06Guang Zhou6 months~5 years community children 1:2 case–control study12.4% (26/209)27.3% (114/418)
[20]20132012/01–2012/06Guang Zhou6 months~5 years community children 1:2 case–control study19.9% (208/1046)30.9% (647/2092)
[21]20142013/02–2013/06Guang Zhou6 months~5 years community children 1:1 case–control study13.1% (164/1250)24.7% (309/1250)
[22]20152013/02–2013/06Guang Zhou8 months~6 years community children1:1 case–control study9.0% (158/1754)19.6% (343/1754)
[23]20162014/02–2014/07Guang Zhou6 months~5 years community children1:2 case–control study14.0% (123/879)21.5% (385/1793)
[24]20192014/02–2014/07Guang Zhou6 months~8 years community children1:1 case–control study12.5% (102/819)23.0% (188/819)
[24]20192015/03–2015/07Guang Zhou6 months~8 years community children1:1 case–control study12.0% (248/2080)14.8% (308/2080)
[24]20192016/03–2016/05Guang Zhou6 months~8 years community children1:1 case–control study7.9% (101/1286)12.0% (154/1286)
[25] *20142012/12–2013/01Bei JingOutpatients of all age groupsTest-negative design2.0% (14/695)4.4% (57/1303)
[26]20182013/11–2014/04Bei JingOutpatients of more than 60 years oldTest-negative design6.8% (9/133)10.2% (36/354)
[27]20162013/12–2015/05Bei JingInpatients of all age groupsTest-negative design11.9% (42/353)12.6% (173/1372)
[28]20172014/11–2014/12Bei Jing6~18 years old students case–control study43.3% (91/210)40.8% (692/1698)
[29]20172014/11–2015/04Bei JingOutpatients of all age groupsTest-negative design4.3% (149/3434)3.7% (215/5863)
[30] *20172015/10–2016/05Bei JingOutpatients of all age groupsTest-negative design13.4% (47/351)11.8% (207/1761)
[31]20182015/11–2016/03Bei JingOutpatients of all age groupsTest-negative design4.2% (124/2969)4.3% (342/8031)
[32]20182016/11–2017/04Bei JingOutpatients of all age groupsTest-negative design2.9% (75/2626)3.6% (278/7816)
[33]20192016/11–2017/04Bei JingOutpatients of all age groupsTest-negative design5.6% (18/322)7.2% (137/1905)
[34]20192016/11–2017/04Bei Jing6~18 years old studentscase–control study17.0% (30/175)36.5% (229/628)
[35]20192016/11–2017/04Bei JingInpatients of more than 60 years oldTest-negative design11.0% (16/145)13.3% (70/528)
[35]20192017/11–2018/04Bei JingInpatients of more than 60 years oldTest-negative design12.8% (19/149)17.1% (56/328)
[36]20192016/10–2017/04Zhe JiangOutpatients of 6months~6 yearsTest-negative design6.1% (17/277)10.9% (41/375)
[36]20192017/10–2018/04Zhe JiangOutpatients of 6months~6 yearsTest-negative design4.0% (5/126)9.9% (38/383)
[37]20162011/10–2012/09Su Zhou6 months~5 years community children 1:1 case–control study4.2% (18/427)11.5% (49/427)
[38] *20182015/10–2016/02Su ZhouKindergarten children aged 3–6 yearsTest-negative design10.3%(17/165) *11.3% (23/203) *
[39] *20182016/10–2017/02Su ZhouKindergarten children aged 3–6 yearsTest-negative design10.6%(17/160) *13.3% (53/398) *
Ref. [25] *: The diagnosis of influenza-positive patients was based on the isolation of influenza viruses from cell cultures; other studies were based on RT-PCR. Ref. [25] * and [30] *: Vaccination information was self-reported; vaccination information of other studies can be found in electronic database of vaccination. Ref. [38] * and [39] *: The original study method was cohort study, but used test-negative design method to analyze, so we showed the positive rate of tested-sample in Table 1.
Table
Table 2. Quality evaluation of eligible studies: based on the Newcastle–Ottawa Scale.
Table 2. Quality evaluation of eligible studies: based on the Newcastle–Ottawa Scale.
[Ref.]Published YearSelectionComparabilityExposureTotal
Scores
Is the Case Definition Adequate?Representativeness of the CasesSelection of ControlsDefinition of ControlsComparability
of Cases and Controls on the Basis of the
Design or Analysis		Ascertainment of ExposureSame
Method of Ascertainment for Cases and Controls		Non-Response Rate
[19]2013101121107
[20]2013101121107
[20]2013101121107
[21]2014101121107
[22]2015101121107
[23]2016101121107
[24]2019101121107
[24]2019101121107
[24]2019101121107
[25]2014100111105
[26]2018100111105
[27]2016100111105
[28]2017101021106
[29]2017100111105
[30]2017100111105
[31]2018100111105
[32]2018100111105
[33]2019100111105
[34]2019101111106
[35]2019100111105
[35]2019100111105
[36]2019100111105
[36]2019100111105
[37]2016101121107
[38]2018100111105
[39]2018100111105
Table
Table 3. Influenza VE subgroup analysis.
Table 3. Influenza VE subgroup analysis.
Study SubjectsHeterogeneity Test: χ2 (P)I2Model SelectionOR (95%CI)VE (95%CI)
Receipt of Two Doses of Vaccine *
Children 6–35 months receiving one dose20.8 (0.0009)76%random-effects model0.55 (0.36–0.82)45% (18–64%)
Children 6–35 months receiving two doses4.76 (0.45)0fixed-effects model0.43 (0.36–0.50)57% (50–64%)
Age Groups
6 months to 8 years37.4 (0.0003)65%random-effects model0.53 (0.46–0.61)47% (39–54%)
More than 60 years5.18 (0.64)0fixed-effects model0.82 (0.67–1.00)18% (0–33%)
Disease Severity **
inpatients11.74 (0.02)66%random-effects model 0.79 (0.56–1.11)21% (−11–44%)
outpatients10.93 (0.03)63%random-effects model 0.87(0.68–1.10)13% (−10–32%)
* According to the influenza vaccine manufacturer prescribing information, children aged 6–35 months should receive two doses of influenza vaccine. ** Compared to outpatients, inpatients were considered more seriously ill.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Yang, X.; Zhao, H.; Li, Z.; Zhu, A.; Ren, M.; Geng, M.; Li, Y.; Qin, Y.; Feng, L.; Peng, Z.; et al. Influenza Vaccine Effectiveness in Mainland China: A Systematic Review and Meta-Analysis. Vaccines 2021, 9, 79. https://doi.org/10.3390/vaccines9020079

AMA Style

Yang X, Zhao H, Li Z, Zhu A, Ren M, Geng M, Li Y, Qin Y, Feng L, Peng Z, et al. Influenza Vaccine Effectiveness in Mainland China: A Systematic Review and Meta-Analysis. Vaccines. 2021; 9(2):79. https://doi.org/10.3390/vaccines9020079

Chicago/Turabian Style

Yang, Xiaokun, Hongting Zhao, Zhili Li, Aiqin Zhu, Minrui Ren, Mengjie Geng, Yu Li, Ying Qin, Luzhao Feng, Zhibin Peng, and et al. 2021. "Influenza Vaccine Effectiveness in Mainland China: A Systematic Review and Meta-Analysis" Vaccines 9, no. 2: 79. https://doi.org/10.3390/vaccines9020079

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop