Real-World Effectiveness of the mRNA COVID-19 Vaccines in Japan: A Case–Control Study

The real-world effectiveness of the coronavirus disease 2019 (COVID-19) vaccines in Japan remains unclear. This case–control study evaluated the vaccine effectiveness (VE) of two doses of mRNA vaccine, BNT162b2 or mRNA-1273, against the delta (B.1.617.2) variant in the Japanese general population in the period June–September 2021. Individuals in close contact with COVID-19 patients were tested using polymerase chain reaction (PCR). A self-administered questionnaire evaluated vaccination status, demographic data, underlying medical conditions, lifestyle, personal protective health behaviors, and living environment. Two vaccine doses were reported by 11.6% of cases (n = 389) and 35.2% of controls (n = 179). Compared with controls, cases were younger and had a lower proportion who always performed handwashing for ≥20 s, a higher proportion of alcohol consumers, and a lower proportion of individuals living in single-family homes or with commuting family members. After adjusting for these confounding factors and day of PCR testing by multivariate logistic regression analysis, the VE in the period June–July (delta variant proportion 45%) was 92% and 79% in the period August–September (delta variant proportion 89%). The adjusted VE for homestay, hotel-based isolation and quarantine, and hospitalization was 78%, 77%, and 97%, respectively. Despite declining slightly, VE against hospitalization remained robust for ~3 months after the second dose. Vaccination policymaking will require longer-term monitoring of VE against new variants.

On the other hand, VE against serious illnesses requiring hospitalization was reported to still be high not only 3 months after the second vaccination but also 6 months after [9][10][11][12][13][14]. The degree of VE varies depending on the age and sex of the target population, underlying diseases, immune attenuation due to the time since the last vaccination, and the status of the primary epidemic strains; however, studies thoroughly examining these factors in Asian countries such as Japan remain limited [15].
Non-pharmaceutical public health measures other than vaccines and personal health prevention behaviors reportedly prevent COVID-19 [16][17][18]. A meta-analysis revealed that the incidence of COVID-19 is negatively associated with handwashing (relative risk [RR]: 0.47, 95% CI: 0.19-1.12), mask-wearing (RR: 0.47, 95% CI: 0.29-0.75), and physical distancing (RR: 0.75, 95% CI: 0.59-0.95) [17]. Randomized trials of the effectiveness of nonpharmaceutical interventions such as wearing a surgical mask against COVID-19 infection also reported preventive effects [19]. However, to the best of our knowledge, no VE studies have adjusted for personal protective health behaviors and preventive measures. Most realworld studies examining VE against COVID-19 have been test-negative case-control and cohort studies using large databases to obtain information regarding vaccination, laboratory testing, clinical infection, and demographic characteristics [15]. If these databases did not include information regarding personal protective health behaviors, the VEs were not adjusted for those behaviors.
In Japan, mRNA vaccination with BNT162b2 and mRNA-1273 has been promoted since February 2021, with priority given to healthcare workers, the elderly, and high-risk individuals. Vaccination targeting later expanded to include generally healthy individuals and those over 12 years of age, and by the end of 2021, approximately 70% of the population of Japan had been vaccinated with two doses [20]. As of February 2022, a booster vaccination campaign for a third dose was being conducted in response to the rapid spread of a new mutant strain designated omicron [20]. Evaluating VE is extremely difficult in Japan because there is no framework in place to collect information on vaccination history, morbidity, and hospitalization at the individual level for use in research. Only a few case-control studies using a test-negative design conducted in fever outpatient clinics in Japan have been reported [21][22][23][24]. Significant protective effects were reported from 14 days after the first vaccination, and the VE against symptomatic COVID-19 was reported >90% after 14 days from the second vaccination dose [21,22]. The VE against delta (B.1.617.2) variant-related symptomatic COVID-19 was slightly lower, at approximately 80% [22,24]. However, disease severity-specific VE has not been evaluated yet. In addition, as the prevalence and mortality rates of COVID-19 in Japan were lower than those in other countries even before the introduction of the vaccine [25,26], a positive effect of lifestyle and protective health behaviors among the Japanese population has been suggested. However, factors such as protective health behaviors have not been considered in previous studies of VE [21][22][23][24].
The purpose of this study was to evaluate VE in Japan, adjusting for confounding factors such as personal protective health behavior in relation to the duration since the last vaccination and the proportion of delta variants during the period studied.

Study Setting and Participants
We conducted a case-control study to evaluate VE against SARS-CoV-2 infection and COVID-19-related hospitalization in Saga Prefecture and Fukuoka Prefecture, both of which are in southern Japan. Cases included symptomatic or asymptomatic SARS-CoV-2-infected patients (≥16 years of age) who were diagnosed by polymerase chain reaction (PCR) test taken at public health centers as part of an epidemiologic survey for close-contact subjects between 4 June and 26 September 2021. This survey period coincided with the end of the fourth wave of COVID-19, driven by the alpha variant, and the entirety of the fifth wave of COVID-19, driven by the delta (B.1.617.2) variant [27]. According to reports from the public health centers in the study area, the proportion of delta variants during June and July was 45%, while that of August and September was 89%. During the study period in Japan, patients with asymptomatic or minor cases of SARS-CoV-2 infection were advised to stay at home or in a hotel for 10 days, and those requiring treatment were admitted to a hospital for at least 10 days. Patients with mild cases were asked to participate in this study when the public health center reported their PCR results, and hospitalized patients were asked to participate when they were discharged. Controls were selected from individuals with negative SARS-CoV-2 PCR test results who were in close contact with cases and either lived in the same household or had been in face-to-face proximity of within 1 m with a case-patient for 15 min without wearing a face mask.
The sample size needed for the study was calculated as follows: assuming α = 0.05/ number of variables (20 items) = 0.0025, β = 0.20, VE = 50-70%, and the proportion of vaccination uptake among controls = 40-60%, the required sample size was 35 to 110 cases and 105 to 330 controls.
Participants provided written informed consent after receiving an explanation of the study purpose and contents and the conditions of cooperation in the study. The study protocol was approved by the Ethics Committee of Saga University (approval nos. R2-39, R3-28).

Data Collection
The following data were obtained using a self-administered questionnaire: sex, date of birth, area of residence, weight, height, and underlying medical conditions (such as respiratory disease, heart disease (including hypertension), kidney disease, liver disease (but excluding fatty liver and chronic hepatitis), diabetes, blood diseases (but excluding iron deficiency anemia), diseases that reduce immune function (including malignancies receiving treatment or palliative care), and others). Participants were also asked about their history of pneumonia, general health condition, smoking history, alcohol drinking history, physical activity, sleep, eating habits, frequency of going out, use of daycare, occupation, education, blood type, family members living together, size of residential space, and whether or not there were any COVID-19-infected people around them. Regarding personal protective health behaviors and measures, we asked whether participants had taken any of the following measures: mask-wearing, washing their hands for at least 20 s upon returning home, use of chlorine-or ethanol-based disinfectants, physical distancing, regular ventilation of the home, avoid eating meals with 5 or more persons, and regularly obtaining COVID-19 information. Vaccinated individuals responded to a questionnaire regarding the vaccine used (Pfizer-BioNTech: BNT162b2 or Moderna: mRNA-1273), lot number, number of doses, and date of vaccination, based on their vaccination certificate. Cases responded to whether they had symptoms of COVID-19 and, if so, the type of symptoms, date and time of symptom onset, duration, and severity. Information about the places of care or stay was obtained from the patient, the city office, and the hospital.

Statistical Analysis
The primary analysis assessed the VE against SARS-CoV-2 infection and hospitalization of at least one dose of either vaccine, two doses, or partial vaccination (only one dose) compared with no vaccination. Subgroup analyses were performed to estimate VE according to duration since the last vaccination dose, severity, and strain-specific protection at the time of diagnosis. We first performed bivariate analyses to assess differences in indicators of background characteristics such as area of residence, date of PCR testing, lifestyle, and personal protective health behaviors between cases and controls using the chisquare test or Wilcoxon rank-sum test. Background characteristic variables that exhibited a p-value < 0.0025 (0.05/20) or appeared to be medically related to COVID-19 and vaccination status were considered potential confounders for adjustment. Multivariable logistic regression models were constructed to calculate odds ratios (ORs) with 95% confidence intervals (CIs). We employed the following continuous and categorical variables for adjustment: sex, age (in 10-year intervals), area of residence, underlying medical conditions, date of PCR test, handwashing for 20 s, current alcohol drinking, residence in a single-family home, and family members commute to work or school. Adjusted VE was calculated as (1-adjusted OR) × 100 (%). Commercial software (ver. 9.4 for Windows; SAS Institute, Cary, NC, USA) was used for all statistical analyses.

Results
During the survey period, we explained the purpose of the study and then mailed questionnaires to 612 potential cases, and a total of 398 (65.0%) responded. A small number of close contacts of the cases tested negative by PCR; thus, 222 subjects became control candidates. We explained the purpose of the study to the potential controls and then mailed each a questionnaire, receiving responses from 179 (80.6%) candidates. In total, we enrolled 398 cases (male: 208, female: 190; mean age: 41.7 ± 14.7 years) and 179 controls (male: 69, female: 110; mean age: 46.9 ± 19.3 years) for this analysis. Table 1 summarizes the characteristic of the study subjects. There were no significant differences in terms of the date of PCR testing or the presence of underlying diseases. In terms of protective health behaviors and living environment, cases were significantly less likely than controls to spend more than 20 s washing their hands, to live in a single-family home, and to have a family member living with them who commuted to and from work or school.  Table 2 summarizes the vaccination status of the study subjects. The proportion of unvaccinated cases was significantly higher than that of controls, and even among those vaccinated, the largest proportion was less than 13 days after the first dose. There was no significant difference in terms of which vaccine was used. Cases diagnosed in June and July were mostly elderly, whereas those diagnosed in August and September were generally younger ( Figure S1).  Table 3 summarizes factors associated with COVID-19 infection. One dose or two doses of COVID-19 vaccine, female sex, washing hands for >20 s each time, living in a single-family home, and having family members who commute to work or school were negatively associated with COVID-19 infection. In contrast, age and alcohol consumption were positively associated with infection. After adjustment for these factors and PCR test date, adjusted ORs for one and two doses of vaccine in June and July, when the proportion of delta variant was 45%, were 0.38 (95% CI: 0.07-2.10) and 0.08 (95% CI: 0.01-0.65), respectively, and adjusted ORs for one and two doses of vaccine in August and September, when the delta variant proportion was 89%, were 0.32 (95% CI: 0.17-0.60) and 0.21 (95% CI: 0.11-0.40), respectively (Table 4). Adjusted ORs for one and two doses of COVID-19 vaccine against COVID-19-related hospitalization were lower than those against COVID-19-related hotel-based isolation and quarantine, or homestay.   Figure 1 shows adjusted values of VE against COVID-19 infection and COVID-19related hospitalization according to time since vaccination. The adjusted VE values against COVID-19 infection were statistically significant after more than 14 days from the first dose, with peak effectiveness occurring less than 13 days from the second dose. In contrast, the adjusted VE values against COVID-19-related hospitalization were statistically significant even less than 13 days from the first dose and increased overtime after the second dose.
More than 14 days from 2nd dose to PCR test 0.21 0.01 0.30 0.01 0.31 0.25 (0.10-0.42) (<0.001-0.15) (0.14-0.65) (<0.001-0.09) (0.10-1.02) (0.11-0.57) * Sex, age (10-year intervals), area of residence, underlying medical conditions, PCR test date, handwashing for 20 s, current alcohol drinking, living in a single-family home, and family members' commute to work or school. Figure 1 shows adjusted values of VE against COVID-19 infection and COVID-19related hospitalization according to time since vaccination. The adjusted VE values against COVID-19 infection were statistically significant after more than 14 days from the first dose, with peak effectiveness occurring less than 13 days from the second dose. In contrast, the adjusted VE values against COVID-19-related hospitalization were statistically significant even less than 13 days from the first dose and increased overtime after the second dose.  Unlike the effectiveness against COVID-19-related hotel-based isolation and quarantine or homestay, VE against COVID-19-related hospitalization did not decline (Table S1). We estimated the adjusted VE against COVID-19 during the time in which the delta was predominant (August and September) according to the place of care or stay. The adjusted VE against COVID-19 with homestay or hotel-based isolation for two doses of vaccine was 74% (95% CI: 50-87%), whereas that against hospitalization was 98% (95% CI: 85-99%). There were no significant differences for aOR against COVID-19 for the age group (<60 and ≥60 years old or vaccine manufacturer (Pfizer and Moderna) (Table S1).

Discussion
In this case-control study conducted from June to September 2021 in Japan, we found that the overall VE of the mRNA COVID-19 vaccine was 65% (95% CI: 39-80%) for a single dose and 81% (95% CI: 65-89%) for two doses. The adjusted VE of two doses against COVID-19 infection in August and September, when the delta variant was predominant, was 79% (95% CI: 60-89%), which was slightly lower than that in June and July (92%, 95% CI: 35-99%) when the proportion of delta variant was less than 50%. Importantly, the adjusted VE of two doses of mRNA vaccine against hospitalization was robust, at 98% (95% CI: 85-99%), even when the delta variant was predominant (August and September). We also examined the VE against COVID-19 in Japan after adjusting for confounding factors, including personal protective health behaviors.
Both asymptomatic and symptomatic mild cases of COVID-19 were included in this analysis. In contrast, case-control studies conducted in Japan using a test-negative design at clinics for fever outpatients during the same period were limited to symptomatic cases [21,23]. Arashiro et al. reported that the VE against symptomatic COVID-19 in August 2021 in a metropolitan area for two doses of mRNA vaccine was 87% (95% CI: 80-91%) [21]. Maeda et al. reported that the VE during the July to September 2021 period in a nationwide setting was 88.7% (95% CI: 78.8-93.9%) [22]. The overall VE of our study was slightly lower, owing to the inclusion of asymptomatic cases; however, the CIs of the present study overlapped with those of other studies conducted in Japan. Thus, we believe that our VE estimates are comparable, despite the differences in sites, study design, and case definitions.
Observational studies have reported lower VE against delta variant COVID-19 infection compared with the alpha variant, especially among vulnerable groups; however, the VE in those studies was still high in the short term after the second vaccine dose [10][11][12]. A systematic review reported that the VE of two doses of the mRNA vaccine against COVID-19 infection was 74% (95% CI: 62-85%) for the delta variant [13]. Our results are in line with previous reports, with a VE against infection after two doses of any vaccine of 79% (95% CI: 60-89%) during the period in which the delta variant was dominant. Although lower than the VE against infection with the alpha variant, this value met the minimum VE of 50% recommended by the World Health Organization [28]. The VE against severe disease requiring hospitalization was quite high, even when the delta variant was dominant. The adjusted VE against COVID-19-related hospitalization was 98% (95% CI: 85-99%) in our study (Table S1), which is similar to the pooled VE against severe outcomes after two doses reported in a recent meta-analysis of 98.5% (95% CI: 95-99%) [29]. These results suggest that the mRNA vaccines help prevent severe COVID-19 breakthrough infection. Some recent studies reported a diminished VE against omicron (B1.1.529) variant COVID-19 infection [30,31] due to the potential for immune evasion, which was supported by in vitro neutralization assays [8,[32][33][34]. In contrast to the VE against the delta variant, the VE against the omicron variant in terms of hospitalization was reportedly lower, at 65% (95% CI: 51-75%), although it remained above 50% [35]. However, it was reported that the VE against hospitalization improved to 86% (95% CI: 77-91%) after three doses of the vaccine [8,35]. Recent studies on SARS-CoV-2 evolutions have reported the intrinsically high recombination and mutation rate in the Spike genes, which may increase binding affinity to human ACE2 receptors [36,37]. Further studies to monitor VE against the omicron variant, which had the highest number of recurrent mutations [37], are needed to understand the role of prevention measures with vaccination and to develop effective vaccination policies.
Several studies have reported the effectiveness of non-vaccine public health measures and personal health protection behaviors in preventing COVID-19 [16][17][18][19]. A systematic review and meta-analysis reported RRs of COVID-19 infection for handwashing, maskwearing, and physical distancing of 0.47 (95% CI: 0.19-1.12), 0.47 (95% CI: 0.29-0.75), and 0.75 (95% CI: 0.59-0.95), respectively [17]. In this study, the OR of handwashing for more than 20 s each time against COVID-19 infection was 0.60 (95% CI: 0.41-0.88), which indicates that personal protective health behaviors are negatively associated with COVID-19 infection. People who take precautions such as thorough hand washing are likely to be protected not only from contact infection but also from other routes of infection. Alcohol consumption was more prevalent in the case group, which may reflect more opportunities for drinking or socializing and, thus, more opportunities for infection. In addition, fewer people in the case group lived in a single-family home or had family members who commuted to work or school, reflecting the fact that many of the cases were younger. Few studies using large databases conducted in other countries have adjusted for confounding associated with protective health behaviors and residential environment. One of the advantages of our study is that we were able to evaluate VE after adjusting for these factors.
Other advantages of this study were as follows. First, the definitions of cases and controls were based on PCR test results, resulting in few misclassifications. Second, vaccination history was confirmed by vaccination certificates, thereby minimizing recall bias. In comparison with previous studies in Japan, our study had two more advantages. We recruited the participants from among subjects who underwent PCR testing after coming into close contact with COVID-19 patients, independent of symptoms; thus, the selection was not biased by medical treatment-seeking behavior or test-seeking behavior of the participants. Owing to this recruiting method, we could estimate VE against infection in real-world situations. Finally, we were able to confirm that a high VE was maintained in cases that required hospitalization in Japan.
Our study also had several limitations. First, we did not collect information regarding the SARS-CoV-2 strain in each case. We collected information regarding the epidemic strain in the study area, and we estimated VE by period according to the proportion of Vaccines 2022, 10, 779 9 of 11 delta variants detected in the area. Second, we could not evaluate VE for longer periods after vaccination because the longest duration was 3 months. Third, we did not have clinical information regarding the severity of each case; instead, we used the settings of care or stay (i.e., hospital, hotel, or home) as surrogates for indicators of severity. Not all hospitalized cases were severe; thus, not all cases needed ventilators or extracorporeal membrane oxygenation in Japan. Fourth, cases involving severe disease or death were not included in the study. This selection bias may have led to an underestimation of VE against severe diseases. Fifth, we could not examine differences in VE according to age, presence of high-risk comorbid conditions, or vaccine manufacturer due to the limited sample size. Finally, protective health behaviors were self-reported. In this study, we found that handwashing was negatively associated with COVID-19; however, reverse causation might have occurred due to the retrospective design.

Conclusions
This case-control study examined the real-world effectiveness of mRNA COVID-19 vaccines among the Japanese population. After adjustment for confounding factors, including protective health behaviors and environmental factors, the VE against COVID-19 infection were 65% (95% CI: 39-80%) for a single dose and 81% (95% CI: 65-89%) for two doses. Although the VE of two doses against infection was slightly lower during the period in which the delta variant was dominant, the VE against hospitalization remained above 90%. Further studies are needed to monitor VE against new variants and the effectiveness of booster doses over time.
Supplementary Materials: The following supporting information can be downloaded at: https: //www.mdpi.com/article/10.3390/vaccines10050779/s1, Figure S1: Age distribution by diagnosed month among cases; Table S1: Adjusted odds ratio (aOR) against COVID-19 according to diagnosis period, age group, and vaccine manufacture by place of care or stay.

Informed Consent Statement:
Written informed consent has been obtained from all participants in the study.

Data Availability Statement:
The data presented in this study are available on request from the corresponding author (M.H.). The data are not publicly available due to privacy concerns.