Evaluating the Effectiveness of COVID-19 Bluetooth-Based Smartphone Contact Tracing Applications
Round 1
Reviewer 1 Report
The authors study the effectiveness of recently developed contact tracing
smartphone applications for COVID-19 relying on Bluetooth to detect contacts. They model the main aspects affecting the performance of these applications. A compartmental model is established to evaluate the efficiency of the applications in terms of controlling future outbreaks and the effort required. The results show that smartphone contact tracing can only be effective when combined with other mild measures able to reduce R0 and that a centralized model is much more effective, requiring an app utilization percentage of about 50% to control an outbreak.
As this is my research area, I would rather concentrate on the epidemic model in this work. I was happy to see that quarantine is modelled in a way which is not very often seen but which is a more precise description of quarantine than the method used in most works, namely, when in fact isolation is modelled instead of quarantine, as in the latter case not only infected/exposed individuals are temporarily separated but also susceptibles who are feared to be exposed. Up to my knowledge, this way of modelling quarantine was introduced by Lipsitch et al., Science, 300(5627), 1966-1970. I do not know whether the authors had this paper as a motivation, but I think this paper should be cited here. On the other hand, I think that the model could be much more realistic if also exposed classes (both quarantined and unquarantined) were introduced.
Some issues about the model which might be improved:
In my opinion, \tau_Q' is a bit misleading notation, looks like a derivative. A different notation, e.g. tilde would be more appropriate.
The normalisation described in lines 352-356 is not clear to me, the notation q is not used before.
A couple of typos: in line 341, a hyphen is used instead a minus sign. Line 344: which have not been tested.
Author Response
Please see the attachment.
Author Response File: 
Author Response.pdf
Reviewer 2 Report
This is a topical and well-written paper, with a clear background outline, good methodological argument and a solid numerical results section. Contributing to the present fight against the COVID-19 pandemic the authors attempt to determine the effectiveness of inter-personal contact tracing using smartphones. It is assumed that the Bluetooth technology permits to register contacts with a deducible precision; here the authors build on a previous study of theirs. Given that, a hypothetical application is outlined in a centralized and decentralized version which may serve to contain the pandemic spread by appropriate quarantineing of smartphone users suspected to have had contact with infected individuals.
Although research in this direction is most desired currently, I do not find the paper innovative enough as to warrant publication in the present form. Regarding the projected application no specific architectures are given and the inferiority of the decentralized version is expected on grounds of more dependence on discretionary actions by the users. This rather unsurprising conclusion is supported by a simple epidemic model, a variation of SIR complemented with quarantine states, where two key parameters of interest are the utilization (percentage of population adopting the hypothetical application) and the frequency of self-checking for exposure, one of the discretionary actions. The model, in particular the way the respective compartments of the population are defined, and the resulting system of ODEs is constructed and solved numerically, is quite standard with the exception of how the quarantine states are accounted for. However, the introduction of the related q parameters around formulae 2 and 3 is somewhat sketchy and confusing (I suggest a revision here, also, items S->Q_T, I->Q_T and Q_I->R on p. 8/9 in my opinion need rewriting); there are confusing or unexplained statements ("the second equation in 4 is negative", "the time has been committed in all the classes", "the whole average tracing time to distribute the tracing quarantine time among the days", "The purpose is that, if the tracing time is long (as in the decentralized model), it is because it takes time to trace back the prior contacts", "obtain the people that has been quarantined by contact tracing" etc.); prime is used in the tau as well as to denote a derivative (which is continuous even though the time is discrete). The presented numerical results are based on arbitrary parameter settings and an approximate model, therefore can well serve as preliminary insight (as mentioned before, not altogether surprising qualitatively), but, not being corroborated by any experimental study, do not provide much useful information. Thus, a tweaking of the presentation of the analytical considerations, and experimental validation with a wider and realistic range of epidemic and contact tracing parameter settings are strongly recommended.
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Reviewer 3 Report
The proposed article is well written, structured and relevant for the time being. Some remarks from my part are:
- There is no description about the detection accuracy of a smartphone. I would add an extra sentence that states the Bluetooth beacon rate.
- It is nice to read the privacy considerations of the different detection approaches. I would bring this more to the foreground of the article.
- I would add an illustration to indicate the differences more visually between a decentralized and centralized approach.
- I would include different precision profiles. Currently, 1 profile is added. I'm curious what the model would estimate when 100%, 75%, 25%, 0% of the real contacts are detected and different false positives.
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Round 2
Reviewer 2 Report
The Authors have improved their text to the extent is now sounds more convincing. The equations, too, look more comprehensible now. I maintain that it is an interesting study - even though it may not offer much in the way of new epidemiological math, it does propose a consistent parametrized model. I agree with the Authors that the model could and should be put to test with other, more realistic parameter settings. At the same time I maintain that complementing this study with some real-world experimentation would give it more persuasive power. I consider this a minor revision, since the Authors state in their response that preparations for suitable experiments are already underway. As an afterthought, referring to Fig. 4 and comparing parts a&b with c&d, one is led to think that the proposed defense becomes quite ineffective as R_0 grows beyond 3, whereas various reports on COVID-19 suggest far larger values than 3 at times. Minor remarks: the passage "The health authorities do not have access to these data, so they do not know exactly which contact has raised this risk notification; hence, in some way, nobody feels pressured as no information is revealed." in the context of fighting an epidemic sounds a little unfortunate and gives an air of helplessness, I don't know if this is what was intended. Maybe a rephrasing would be in order. Also, parts e) and f) of the caption to Fig. 1 suggest, contrary to the drawing, that only one and the same key is uploaded and downloaded.
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
