Analyzing the Effectiveness of COVID-19 Lockdown Policies Using the Time-Dependent Reproduction Number and the Regression Discontinuity Framework: Comparison between Countries

: This study compares the effectiveness of COVID-19 control policies on the virus’s spread and on the change of the infection dynamics in China, Germany, Austria, and the USA relying on a regression discontinuity in time and ‘earlyR’ epidemic models. The effectiveness of policies is measured by real-time reproduction number and cases counts. Comparison between the two lockdowns within each country showed the importance of people's risk perception for the effectiveness of the measures. Results suggest that restrictions applied for a long period or reintroduced later may cause at-tenuated effect on the circulation of the virus and the number of casualties.


Introduction
COVID-19 is an infectious disease caused by SARS-CoV-2, which has been declared a global public health emergency [1]. As of 29 May 2021, it has affected more than 100million people and resulted in more than 3.5 million deaths globally (WHO). Governments worldwide have implemented similarly strict containment and closure policies to mitigate the pandemic in order to limit the spread of the virus. These restrictive community measures that limit activities or access to resources, facilities, or institutions have been often referred to as "lockdown" measures in Asia, Europe, and America [2,3]. Countries exhibited 'herd behavior' in response to COVID-19 [2] meaning they applied similar restrictive measures. However, the effectiveness of these measures has been different between countries. Previous studies showed that containment measures implemented in countries like China and South Korea have reduced new cases by more than 90%, which has not been the case in many other countries such as Italy, Spain, and the United States [4]. The effectiveness of the social distancing measures was evident in the data of Italy, Germany, and Turkey, but not clearly in the data of the USA and the U.K. [5]. Thus, the public administration community needs to embrace international and comparative perspectives on COVID-19 to inform how governments respond to the crisis, to learn the lessons from more successful governments, and to advance pandemic crisis management [6]. Up to now and currently, the situation is still uncertain, even though the COVID-19 vaccine is being used at full throttle in vaccination campaigns.
Related research shows that policy effectiveness is associated with income groups [2], regional political trust, and compliance [7], as well as country preparedness, socioeconomic factors [4], and a country's values [3]. More and more research has pooled coronavirus data Eng. Proc. 2021, 5, 8 2 of 10 and control measures from countries and regions to compare the effectiveness of public health measures. These studies revealed the requirements needed to enhance scientific analysis and epidemic modeling, and the social and institutional challenges of operating in a global crisis [8]. The USA and Germany both are highly affected countries, with 34.1 million and 3.6 million confirmed cases, ranking 1st and 10th worldwide, respectively, as of 1 June 2021. Austria shares aspects of culture with Germany, and had 0.64 million confirmed cases as of the same date, ranking 38th worldwide. Nevertheless, the pandemic spread patterns of Austria and Germany have been different, especially in the second wave [9]. As for China, The Lancet recognized the quick containment of COVID-19 in China, which sets an encouraging example for other countries [10]. Moreover, these four countries have experienced the whole COVID-19 period, with at least two waves of outbreak, which could help to indicate the long-term effects.
Abundant time series data have been collected, and time-dependent statistical analysis has been widely applied in the public health policy research. Study on Africa used the interrupted time series analysis (ITSA) to analyze the effect of border closure on COVID-19 incidence rates (IRs), which revealed that the implementation of border closures within African countries had minimal effect on the IRs of COVID-19 [11]. The research conducted in England shows that mental health service delivery underwent sizable changes during the first national lockdown by using regression discontinuity in time design (RDiT) [12]. Furthermore, the regression discontinuity design (RDD) has been used by Chinese researchers for examining the lockdown policy effects on air quality, which explores the relationship between anti-epidemic measures and air quality based on the daily data from 326 prefecture-level cities in China [13] and an early assessment with cross-national evidence on the causal impacts of COVID-19 on air pollution by using a RDD approach [14].
This study intends to assess the effectiveness of lockdown COVID-19 control policies on the virus's spread and on the change of the infection dynamics over a year with the event of a resurgence of cases, which have been implemented in China, Germany, Austria, and the USA based on real-time monitoring data and government responses. In this analysis, the different pandemic waves and the characteristics between countries are addressed. This comparative analysis aims to provide important lessons to be learnt from the experiences of these countries. Although the future of the virus is unknown at present, countries should continue to share their experiences, shield populations, and suppress transmission to save lives. In the assessment, the Oxford COVID-19 Government Response Tracker (OxCGRT) was used, which has been also used widely during the pandemic to measure the policies. We focus on the part of containment and closure, including school closing, workplace closing, cancelling public events, restrictions on gathering size, closing public transport, stay-at-home requirements, restrictions on internal movement, and restrictions on international travel. More details can be seen in Reference [15]. Since the vaccination program has been well underway since early 2021, there is hope for a gradual return to normal interaction. However, the virus in different forms poses an ongoing threat. Therefore, we should learn from the knowledge and lessons generated in the lockdown period in order to leverage better public policy to enable more resilient and effective public health services.

Data Sources
Data used in this analysis are from 1 January to 31 December 2020. We obtained data on policy interventions from the Oxford COVID-19 Government Response Tracker (OxCGRT), which has tracked national government policy measures in response to the COVID-19 pandemic globally for 186 countries, starting from 1 January 2020 (Version 7.0). The database details are described in the working paper [16]. Our main interest is lockdown at the city/country-level, such as stay at home orders and restrictions on movement. Data on COVID-19 daily reported cases were obtained from various official sources, including the European Centre for Disease Prevention Control (ECDC), the Johns Hopkins University Centre for Systems Science and Engineering (JHU-CSSE) and the Center for Disease Control and Prevention (CDC) [17].

Epidemics and Regression Discontinuity in Time (RDiT) Model
We used a regression discontinuity in time (RDiT) design to estimate the effectiveness of lockdown policy interventions. RDiT is extended by the regression discontinuity (RD) framework that has applications in several fields. Compared to the standard RD framework, RDiT has been adapted to applications where time is the running variable and treatment begins at a particular threshold in time. In other words, it uses time as the running variable, with a treatment date as the threshold. This approach is close to quasi experimental framework (pre-intervention compared to post-intervention). Papers using RDiT span fields that include public economics, industrial organization, environmental economics, marketing, and international trade [18].
The effectiveness of intervention measures is measured by two ways: real-time reproduction number (R t ) and counts of cases. R t was estimated by the 'earlyR' epidemic model, which is a simplified version of the model introduced by Anne Cori et al. [19]. Parameter estimates were obtained from the early transmission dynamics in Wuhan, China of COVID-19 project by the China Center for Disease Control and Prevention (CCDC), and the serial interval distribution had a mean (±SD) of 7.5 ± 3.4 days (95% CI, 5.3 to 19) [20]. Since the policy interventions may not have immediate effects, we hypothesized a 14-days lag time for counts of cases to coincide with the approximate incubation period of COVID-19.
We took advantage of the pandemic-induced lockdowns as an exogenous policy shock and attempted to retrieve the impact of policy interventions using RDiT approaches. In this approach, we assume the lockdown's start date is when the first "stay at home requirements" become equal to "2", which means to mandate not leaving the house with exceptions for daily exercise, grocery shopping, and 'essential' trips. Alternatively, this was also evaluated as when "restrictions on internal movement" become greater than zero, which means it is recommend not to travel between regions/cities. The usual RDiT regressions were run, both using a polynomial approach and a local linear approach. The equation is as follows: where the outcome variables Y(R t or counts of cases) in country i on date t, Y it , is regressed by treatment variable L it , a dummy variable for pre/post-intervention, a vector of covariates X it , and a flexible nth-order polynomial in f (d it ), and d it denotes the number of days from lockdown date. The coefficient of interest, β i , is the treatment effect of the lockdown interventions on outcome variables in country i. In other words, this is the expected difference between the outcome variable before and after the lockdown. Additionally, α i denotes the country fixed effects and ε it denotes the error term. As countries implemented more than one lockdown because of the secondary COVID-19 waves, we define the first lockdown as the timing of "stay at home requirements" policy adoption (score becomes equal to "2") and the second lockdown as the timing of reimposition after subsequent policy easing. Table 1 presents two consequent lockdowns and summarizes the information about the lockdowns in the case study countries during the research period from 1 January to 31 December 2020, i.e., the date on the lockdown, number of COVID-cases on that day, and the policy stringency index. Policy stringency index is one of the composite measures, which combine different indicators into a general index. The details can be found in the codebook [16]. Although there is a lack of information on policy implication and demographic or cultural characteristics, the value and purpose of the indices is to allow for cross-national comparisons of government interventions.

Estimates of R t
The estimated R t for all included countries (China, Germany, Austria, and the United States) from 1 January to 1 December 2020 are shown in Figure 1. It is clear that all countries were affected by the pandemic after March 2020, and the changing dynamics of the impacts in the four selected countries was different. China had the highest reproduction number, while the maximum reproduction number in the other three countries seemed to be similar. In the early stages, all countries were exposed to extremely high pandemic risk spread rate, which is indicated in Figure 1, with the highest position of the parameter R t in all countries. In the period from March to April, R t gradually declined because of governmental intervention policies to reduce the pandemic spread.
Among the intervention measures, the lockdowns are perhaps the most stringent. Table 1 shows that lockdowns in different case study countries were introduced differently. In China, the first lockdown was implemented on 1 February, i.e., the earliest date of the four studied countries. In Austria and the USA, the first lockdown started almost simultaneously, then followed by the lockdown on 21 March in Germany. From March to May, the curve of R t was flattened; however, it still fluctuated around R t = 1. It is visible from Figure 1 that after improving the situation as a result of the first lockdown, all four countries experienced repetitive rush increases of the parameter R t in different subperiods during March and December 2020. This can be explained by the fact that in each country the removal of the lockdown led to the return of the highly epidemic situation because of the insufficient natural immunity among the population.   sporadic outbreaks of scattered infections or to experience regional outbreaks. For the USA, there was an initial infection peak in April, and the rate of new cases dropped somewhat after the containment interventions. However, it is more of a plateau, and the next peak came in July. Experiencing the temporarily declining, the second wave bounced, increasing exponentially after September. For Germany and Austria, the second wave also came after July, but Austria seemed to control it better.   Especially the Rt shows significant discontinuity for these four countries, and it implies the lockdowns have effects on the spread. Every country had a tendency to flatten Rt after the lockdown. However, the decreasing trend of Rt before d = 0 indicates that some of governmental intervention measures were already implemented before the complete lockdown, such as "keep distance" or "wear masks". As for the daily cases, Germany, Austria, and the USA show a closer discontinuity gap. This can be attributed to the limited cases before the first lockdown time.

Overall Impact of Lockdown Interventions
Most RDiT models were of good fit. The country-specific linear interaction and quadratic interaction regression results are presented in Table 2 (dependent variable is Rt) and Table 3 (dependent variable is daily cases). For China, the Rt could decline by 0.988 before the first lockdown, and the lockdown brought a 4.457 decrease, which is strongly statistically significant. The quadratic interaction regression results were similar, with a 4.432 decrease. For Germany, compared to the slightly increase, the Rt declined by nearly 2 after the lockdown. Austria showed a 1.201 increase before the first lockdown, while there was  Figures 3 and 4 show the regression discontinuity in time estimates, including R t and daily cases. The horizontal axis displays days before and after the complete lockdown at d = 0, the vertical axis defines the value of R t (or daily cases) in the respective day on the horizontal axis. Both show the prima facie evidence of impacts of the first lockdowns. Especially the R t shows significant discontinuity for these four countries, and it implies the lockdowns have effects on the spread. Every country had a tendency to flatten R t after the lockdown. However, the decreasing trend of R t before d = 0 indicates that some of governmental intervention measures were already implemented before the complete lockdown, such as "keep distance" or "wear masks". As for the daily cases, Germany, Austria, and the USA show a closer discontinuity gap. This can be attributed to the limited cases before the first lockdown time.

Overall Impact of Lockdown Interventions
Most RDiT models were of good fit. The country-specific linear interaction and quadratic interaction regression results are presented in Table 2 (dependent variable is R t ) and Table 3 (dependent variable is daily cases). For China, the R t could decline by 0.988 before the first lockdown, and the lockdown brought a 4.457 decrease, which is strongly statistically significant. The quadratic interaction regression results were similar, with a 4.432 decrease. For Germany, compared to the slightly increase, the R t declined by nearly 2 after the lockdown. Austria showed a 1.201 increase before the first lockdown, while there was a 3.831 decrease after the first lockdown. For the U.S., the R t could increase by 1.879, while there was a 5.566 decrease after the first lockdown.
As for the analysis of daily cases, the results looked different. For China, the daily cases would increase 2782.4 if there was no intervention of lockdown. The first lockdown decrease of 3274.5 daily cases was strongly statistically significant. However, for Germany, Austria, and the USA, the situation was different. Even though the R t flattened, the daily cases increased after the first lockdown, with 7162.5, 1473.1, and 37,561.1, respectively. It seems strange but is in line with the facts. there was a 5.566 decrease after the first lockdown.
As for the analysis of daily cases, the results looked different. For China, the daily cases would increase 2782.4 if there was no intervention of lockdown. The first lockdown decrease of 3274.5 daily cases was strongly statistically significant. However, for Germany, Austria, and the USA, the situation was different. Even though the Rt flattened, the daily cases increased after the first lockdown, with 7162.5, 1473.1, and 37,561.1, respectively. It seems strange but is in line with the facts.  Eng. Proc. 2021, 5, 8 6 of 10 a 3.831 decrease after the first lockdown. For the U.S., the Rt could increase by 1.879, while there was a 5.566 decrease after the first lockdown.
As for the analysis of daily cases, the results looked different. For China, the daily cases would increase 2782.4 if there was no intervention of lockdown. The first lockdown decrease of 3274.5 daily cases was strongly statistically significant. However, for Germany, Austria, and the USA, the situation was different. Even though the Rt flattened, the daily cases increased after the first lockdown, with 7162.5, 1473.1, and 37,561.1, respectively. It seems strange but is in line with the facts.    Note: * p ** p *** p < 0.01.

Comparative Effectiveness of First and second Lockdown
As discussed earlier, many countries implemented more than one lockdown because of the secondary COVID-19 waves. Therefore, we took the second wave into the consideration in our research. We compared the Rt and daily cases of 25 days before and after

Comparative Effectiveness of First and second Lockdown
As discussed earlier, many countries implemented more than one lockdown because of the secondary COVID-19 waves. Therefore, we took the second wave into the consideration in our research. We compared the R t and daily cases of 25 days before and after each lockdown. The country-specific quadratic interaction regression results are presented in Table 4 (dependent variable is R t ) and Table 5 (dependent variable is daily cases).
For China, the effectiveness of the second lockdown was weaker compared to the first lockdown, with an estimated R t decrease by 1.556 in the first lockdown and increase by 0.585 in the second lockdown. Meanwhile, the daily cases did not decrease as fast as before (−24.5 vs. −762.3). In Germany, we found out the effect on the R t was slightly stronger; R t decreased by 0.715 (from 0.827 to 0.112), although it still was positive. Therefore, there was a significant increase in the daily cases after the second lockdown (4811.651 increase). For Austria, the effect on R t after the first lockdown and the second lockdown was a 0.564 increase and 0.128 decrease, respectively. It means that the second lockdown contributed to flattening the R t curve. Meanwhile, we also saw a decrease of daily cases in Austria, with 108.633 and 254.206, respectively. For the USA, there was a significant increase of R t after the second lockdown, which had a 4.75 increase. Compared to the first lockdown, the situation became worse, with a higher R t (1.334 vs. 4.750) and daily cases (−2058.49 vs. 2100.23).
It is noted that the results were related to the baseline, namely, the total confirmed and infected cases. For China, the first outbreak was the most serious wave during the COVID-19 period, which affected the country nationwide. Therefore, the first lockdown quickly smoothed the curve and reduced a large number of cases. The second lockdown was introduced at regional level to smooth provincial outbreaks. On the other hand, for Europe and the USA, the pattern was different. The first lockdown in Europe and the USA was introduced when the cases were growing and the epidemics' epicenters were detected in neighboring countries. The second lockdown was implemented to deal with the domestic outbreak.

Discussion
We offer a retrospective study that provides cross-national evidence on the causal impacts of policy intervention on COVID-19 spread. A rich database was assembled from various sources, which were analyzed with EarlyR and RDiT models. Overall, the results show that COVID-19-induced lockdowns resulted in a decrease in R t and daily cases, which varied across different countries. We expected the lockdown could mitigate the spread of COVID-19, but the results were not satisfactory and should be further explained. Comparing different countries, China had the most effective lockdown, which could lower the R t and decrease the daily cases, while the USA, Germany, and Austria had strongly decreased R t but presented large daily case enhancement. Comparison between the two lockdowns within each country showed that people's risk perception was relaxed during the second lockdown, especially in Germany (the increased daily cases were the highest in the studied countries) and the USA (the increased reproduction number was the highest in the studied countries).
Our results were similar to the relevant research, which suggested that the stringent lockdown policies adopted in China, Italy, and Spain were among the most effective national-scale policies [21]. China also showed the most effective results in our study. For Germany and Austria, they showed different patterns in our study, although they share common borders in Central Europe and have substantial cultural, historical, and economic ties [22]. The differences may be explained in terms of the fact that the power was consolidated in central governments in Austria, while in Germany, states retain their autonomy [22]. The USA presented the most unexpected results, and some research gives further explanations: namely, except for the timing and strictness of implementing measures [23,24], national culture, economic, and health and social issues also influence the results [25,26]. Most importantly, our results suggest that restrictions applied for a long period or reintroduced late in the pandemic would exert, at best, a weaker, attenuated effect on the circulation of the virus and the number of casualties. Our results support the conclusion of Haug et al. (2020) that lockdowns should be strict and brief [27].