Influenza Vaccine Effectiveness in Korea: A Systematic Review and Meta-Analysis of Real-World Evidence from the Past Decade
Round 1
Reviewer 1 Report
Comments and Suggestions for Authors2.3 You mention that data was collected on a pretested, standardized form. How was this validated? Have you considered publishing this form or has this form been used for prior published research?
2.4 I would recommend to amend this paragraph to indicate that while the models suggest a reasonably well-balanced publishing of findings, the possibility still exists for overestimation of effect because it is impossible to know what never made it to press. Additionally, you say the same thing in 3.3, so a) why does this appear twice; and b) I would recommend to revise as per 2.4 (or delete one of the sections).
Table 1: The table is missing a full caption for the abbreviations. Also, what is the difference between a RIDT, RAT, and influenza rapid test? Also, usually, we just call the EIAs "RDTs," rather than including the pathogen in the name (e.g. "RDT" or "RDT for influenza," (depending on the context), rather than "RIDT").
3.3 See 2.4.
Sections 2.5 and 3.4 are also highly redundant.
Discussion
Meta analyses are inherently fraught. You are pooling lots of very heterogenous studies (variable vaccine technology, variable pathogen subtypes) that individually generally lack statistical significance, and hoping that by combining it, it generates something that becomes statistically significant, which is a bit of a flawed assumption, but you've done the best you can with the available data.
It would be useful to include emphasis of clinical significance, and not just statistical significance in your discussion of the data. Specifically, it would be useful to discuss at what threshold of VE correlates to meaningful reductions in costs, as well as hospitalizations, if you have any of that national data.
Author Response
Q1. 2.3 You mention that data was collected on a pretested, standardized form. How was this validated? Have you considered publishing this form or has this form been used for prior published research?
Thank you for your comment. We originally used the term “pretested standardized form” to indicate that the data extraction sheet was developed jointly by both authors through consensus and used consistently throughout the study. However, we agree that this term may imply a more formal validation process. Therefore, we have revised the sentence as follows:
(Page 3, Line 97-98, Manuscript) Data extraction was performed separately by both reviewers using a standardized form that was jointly developed through consensus.
Q2. 2.4 I would recommend to amend this paragraph to indicate that while the models suggest a reasonably well-balanced publishing of findings, the possibility still exists for overestimation of effect because it is impossible to know what never made it to press. Additionally, you say the same thing in 3.3, so a) why does this appear twice; and b) I would recommend to revise as per 2.4 (or delete one of the sections).
We sincerely thank the reviewer for this insightful comment.
We agree that although statistical tests did not indicate strong evidence of publication bias, the possibility of overestimation cannot be completely excluded, as unpublished studies are inherently unobservable. Accordingly, we have revised the Results section to reflect this important caveat.
Regarding the apparent duplication, Section 2.4 describes the methods used to assess publication bias, whereas Section 3.3 presents the corresponding results. To avoid potential misunderstanding, we have revised the subtitle of Section 3.3 as follows:
(Page 10, Line 179, Manuscript) Results of publication bias assessment
In addition, we have incorporated this concern into the Discussion section by adding publication bias as a study limitation, noting that despite non-significant statistical tests, the possibility of overestimation cannot be entirely ruled out due to the limited number of studies and the inherent inability to account for unpublished data.
(Page 12, Line 265-267, Manuscript) Finally, although statistical tests did not suggest substantial publication bias, the limited number of included studies and the inability to account for unpublished data mean that the possibility of overestimation cannot be entirely excluded.
Q3. Table 1: The table is missing a full caption for the abbreviations. Also, what is the difference between a RIDT, RAT, and influenza rapid test? Also, usually, we just call the EIAs "RDTs," rather than including the pathogen in the name (e.g. "RDT" or "RDT for influenza," (depending on the context), rather than "RIDT").
We sincerely thank the reviewer for the careful evaluation of Table 1.
First, we agree that the abbreviation legend was incomplete. We have revised Table 1 to include a comprehensive list of all abbreviations in the footnote.
Second, we acknowledge that the terminology used for rapid influenza testing (e.g., “RIDT,” “RAT,” and “influenza rapid test”) may have caused confusion. These terms were originally reported as such in the included studies; however, to improve clarity and consistency, we have standardized the terminology throughout Table 1 and the manuscript to “RDT” (rapid diagnostic test).
(Page 8, Line 154-158, Manuscript) TND, test-negative design; ER, emergency room; ILI, influenza-like illness; RDT, rapid diagnostic test (rapid antigen-based test for influenza; terminology such as “RIDT,” “RAT,” and “influenza rapid test” used in the original studies was harmonized as RDT; PCR, polymerase chain reaction (terminology used in the original studies, including “real-time PCR,” was standardized as PCR); M, male; F, female; VE, vaccine effective-ness.
To ensure conceptual accuracy, we have clarified at first mention in the Methods section that RDT refers specifically to rapid antigen-based diagnostic tests for influenza, in contrast to PCR assays.
(Page 3, Line 124-125, Manuscript) Because most studies used rapid antigen-based diagnostic tests for influenza (hereafter referred to as RDTs) rather than polymerase chain reaction (PCR) assays for case confirmation,
Q4. 3.3 See 2.4.
We thank the reviewer for this comment. As described in our response to Comment 2, Section 3.3 has been revised accordingly.
Q5. Sections 2.5 and 3.4 are also highly redundant.
We thank the reviewer for this helpful comment.
Section 2.5 describes the methodological approach used to assess risk of bias, including the ROBINS-I tool and its evaluation domains, whereas Section 3.4 presents the results of that assessment. However, we acknowledge that certain phrases may have appeared repetitive. To address this concern, we have streamlined Section 3.4 to focus exclusively on the results of the risk-of-bias assessment and removed redundant methodological descriptions.
(Page 10, Line 124-125, Manuscript) The following sentence has been removed: The risk of bias across the included studies was assessed using the ROBINS-I tool.
Q6. Discussion
Meta analyses are inherently fraught. You are pooling lots of very heterogenous studies (variable vaccine technology, variable pathogen subtypes) that individually generally lack statistical significance, and hoping that by combining it, it generates something that becomes statistically significant, which is a bit of a flawed assumption, but you've done the best you can with the available data.
It would be useful to include emphasis of clinical significance, and not just statistical significance in your discussion of the data. Specifically, it would be useful to discuss at what threshold of VE correlates to meaningful reductions in costs, as well as hospitalizations, if you have any of that national data.
We thank the reviewer for this important comment. We agree that meta-analyses of heterogeneous studies require cautious interpretation. The primary objective of this study was to conduct a systematic review of real-world influenza VE studies in Korea over the past decade and to estimate pooled effectiveness across age groups and virus types.
Our pooled VE estimates (23.6–32.4%) were consistently lower than those reported in recent global meta-analyses (approximately 34–49%). While explicit VE thresholds linked to hospitalization or cost reductions could not be evaluated due to limitations of the available data, such analyses were beyond the scope of the present study.
Reviewer 2 Report
Comments and Suggestions for AuthorsLines 38-40. The World Health Organization (WHO) estimates that influenza causes 3–5 million cases of influenza-related severe illness and 290,000–650,000 respiratory deaths globally each year, underscoring its substantial public health burden
Comment: add when?
Line 49. Provide a citation.
Lines 75-76. Searches were performed in PubMed, Embase, and Web of Science. Was the search covered by Scopus? Was the search only for English literature or others as well?
Line 198. When compared with findings from other studies.
Provide a citation.
Author Response
Q1. Lines 38-40. The World Health Organization (WHO) estimates that influenza causes 3–5 million cases of influenza-related severe illness and 290,000–650,000 respiratory deaths globally each year, underscoring its substantial public health burden Comment: add when?
We thank the reviewer for this helpful comment. The cited figures are based on the WHO webpage on the global burden of influenza, which was posted in March 2024. To provide clearer temporal context, we have revised the manuscript.
(Page 1, Line 37, Manuscript) According to the World Health Organization (WHO) 2024 update on the global burden of influenza, influenza causes an estimated 3–5 million cases of severe illness and 290,000–650,000 respiratory deaths worldwide each year, underscoring its substantial public health burden [3].
Q2. Line 49. Provide a citation.
We thank the reviewer for this comment. To avoid redundancy and ensure that statements are directly supported by data, we have removed the initial general statement and retained the sentence supported by Ref. 8.
(Page 2, Line 49, Manuscript) The following sentence has been removed: In Korea, influenza remains a substantial health and economic.
Q3. Lines 75-76. Searches were performed in PubMed, Embase, and Web of Science. Was the search covered by Scopus? Was the search only for English literature or others as well?
We thank the reviewer for this important comment.
The literature search was limited to studies published in English. As specified in Section 2.2 (Eligibility criteria), studies not written in English were excluded.
(Page 2-3, Line 87-90, Manuscript) The exclusion criteria were as follows: (1) studies available only as abstracts, case re-ports, case series, or review articles; (2) randomized controlled trials (RCTs); and (3) studies not written in English.
Regarding database coverage, the search was conducted in PubMed, Embase, and Web of Science. Web of Science provides broad multidisciplinary coverage similar in scope to Scopus, while PubMed and Embase offer extensive indexing of medical, public health, and pharmacological literature. Given the substantial overlap in coverage among these major databases, we believe that the majority of relevant studies were captured.
Q4. Line 198. When compared with findings from other studies. Provide a citation.
We thank the reviewer for this comment. We agree that the introductory comparison statement should be directly supported by citation. Accordingly, we have revised the sentence to include references 26 and 27.
(Page 11, Line 201-202, Manuscript) When compared with findings from other studies, the VE estimates observed in this study appear lower [26-27].
