Benford Law to Monitor COVID-19 Registration Data. Comment on Farhadi, N.; Lahooti, H. Forensic Analysis of COVID-19 Data from 198 Countries Two Years after the Pandemic Outbreak. COVID 2022, 2, 472–484

In a recent study published in COVID by Farhadi & Lahooti, 2022 [1], the authors claim that the work carried out by Morillas-Jurado et al., 2022 [2] is an example of a poor application of Benford Law (BL) in the context of pandemic. Morillas-Jurado et al., 2022 examined the COVID-19 epidemic specifically during the first wave of the pandemic and document anomalies in the data that occurred in six Spanish regions.

The main argument to Farhadi & Lahooti’s criticism is that the regional data sets were too small to assess the conformity of COVID-19 data to BL.

However, in Morillas-Jurado et al., 2022, in order to avoid the sample size problem, we ran a sensitivity analysis just to verify that our decision about the conformity of the BL applied also for our sample size. Specifically, in the Methodology section, within Section 2.3, “Sensitivity Analysis Steps”, we explain in detail the steps of the sensitivity analysis devoted to validating our results for a specific number of observations, n.

In particular, as developed in step 2 of the sensitivity analysis, we artificially generated an arbitrary number of perturbations for each series obtained through a Monte-Carlo simulation; specifically, 1000 perturbations for each series.

Therefore, given the observed series $\left\{x_1^{obs},x_2^{obs},x_3^{obs},\dots x_n^{obs}\right\}$, and assuming that the observed values could be not correct, we obtained the i^th series modified as $x_k^i=x_k^{obs}\left(1+u_k^i\right), k=1,\dots ,n$ where $i=1,\dots .1000, {\left\{u_k^i\right\}}_{k=1}^n$ are n values obtained by simulation from the distribution U [−0.1; +0.1]. The U distribution introduces equal variability in all directions.

Then, we moved on the next step in which the BL test is applied to each series $i_{0, } {\left\{u_k^{i_0}\right\}}_{k=1}^n$ generated synthetically, by calculating the statistics distance of χ² and the p-value test for that series. Then, we obtained 1000 synthetic series, with 1000 p-values ${\{p_1^i,p_2^i,\dots ,p_n^i\}}_{i=1}^{1000}$ and 1000 χ² distances.

Summarizing, given a observed series, {$x_k^{obs}{\}}_{k=1}^n$, we can generate 1000 synthetic simulations {{x_ik${\}}_{k=1}^n{\}}_{i=1}^{1000}$ and 1000 p-values ${\{p_1^i,p_2^i,\dots ,p_n^i\}}_{i=1}^{1000}$, then, we calculate both the average p-value $\overline{p}$ and the average distance χ², in addition we calculate quantiles of α-order for those p-values, $q_{\alpha }$. Moreover, from $q_{\alpha }$, it possible to obtain the equivalent to a confidence interval that allow validation about the decision of the BL fulfilment with the observed data.

Farhadi & Lahooti, 2022 also point out, as a weakness of the aforementioned work, that we did not provide precise figures on the observed frequencies.

In fact, the figures of observed frequencies for each region were not provided in the paper to avoid redundancies. The observed frequencies at a national level and for each region were both shown graphically in Figures 2 and 3, respectively, where we compared observed frequencies with the Benford’s Law in order to graphically show anomalies in the data. In addition, as we mentioned in Section 3, “Data and Sources”, we ran our analysis with “datadista Git-Hub repository” data [3], which is publicly available.

Finally, Farhadi & Lahooti, 2022 also commented that epidemic management limitations at the beginning of the outbreak that may have affected the distribution of leading digits. The authors stated that we ignored such limitations.

However, the literature supports our methodology design. The suitability of the BL for monitoring the registration of pandemic data has been tested for many countries. Several papers in the field show how BL is useful, especially when the number of cases is increasing; however, when the number of cases stabilizes, BL is no longer useful [4,5,6,7,8]. Therefore, the period selected for our analysis (March to June 2020) should fit the assumptions for applying this methodology.

Furthermore, the period selected for our analysis (the lockdown during the first wave) allows us to analyse a situation in which the confinement rules were homogeneous for the whole of Spain. Afterwards, decisions on socio-sanitary standards were taken by each of the regions, leading to non-comparable situations.

In addition, within the discussion section, a reflection on the complexity of managing an unexpected pandemic of the magnitude of COVID-19 has been added, especially for a country like Spain where health responsibilities are decentralized among regions and, therefore, adopting common tools to monitor data registration is required.