The Role UX Design Attributes Play in the Perceived Persuasiveness of Contact Tracing Apps

Round 1

Reviewer 1 Report

Authors report on an online study conducted on Amazon 12 Mechanical Turk among 446 Canadian and American residents to investigate the UX design attributes associated with the perceived persuasiveness of Contact Tracings Apps (CTAs). The investigated attributes are the following : perceived usefulness, perceived ease of use, perceived compatibility, perceived enjoyment, perceived trustworthiness, perceived data privacy, and perceived data security.

The topic is relevant and timely. 

To carry out the analysis, they evaluated two versions of three selected displays of the COVID Alert app (No exposure status, Exposure status, Diagnosis report), with and without “persuasive elements”.

The paper has both merits and flaws that have to be addressed.

The paper is well written and organized. The research is well motivated, related work is adequately reported, and the original contribution wrt to existing literature is clearly identified. Analysis of results is neatly conducted.

The study has a meta-analysis / guideline-oriented nature, leaving it open to the designer to determine which persuasive strategy (and which actual interface elements) could provide perceived usefulness, perceived ease of use, etc to the app. Per se, this is not a criticism: it is certainly useful to have guidelines, and it is certainly important to provide new guidelines in new application domains, as in the case of CTAs.

Anyhow, it has to be observed that results are more or less as expected, actually confirming what a UX/HCI expert could have actually foreseen based on general UI design guidelines. On the other hand, it is interesting to have measures of the selected UX design attributes and to see the interdependency of UI elements and the necessity of trade-off.

What bother me more (and that makes the paper weak in its present version) is:

1.     the lack of discussion/justification as to the persuasive strategies/features/messages adopted for the persuasive version of the studied app, since their effectiveness is the building block of the entire study and not effective design choices could be a huge bias for the achieved results (the impact of such design choices is analyzed only for the Diagnosis reporting interface – page 12). This is a key point and authors themselves recognize it on page 10: “The cold start problem is the inability for a persuasive app to know what persuasive strategies or messages that will be effective in changing a first user’s behavior”, even if the proposed approach (“This means by asking first-time users few UX-related questions prior to using the app, their responsiveness to certain persuasive strategies and messages may be predicted”) is quite superficial (robust UX design requires much more than asking first-time users few UX-related questions …).

2.     Generalizability of results, given the narrowness of a study conducted on few interfaces a specific persuasive version of a single specific app.

The "low" scores given to "significance of content",  "scientific soundness" and "overall merit" are related to such flaws.

I think that authors should address Point 1 by clearly and extensively discussing and motivating their design choices to allow the evaluation of their soundness and of the soundess of the overall study (authors refer to a pre-print version of the paper that should more extensively report the study design; since there is no limitation on the length of the paper there is no reason for not discussing design information also in the paper submitted for publication), and Point 2 by clearly resizing the scope of the study and clearly relating the achieved results to the specific case study.

Minor remark:“perceived persuasiveness” should be defined the first time it is mentioned (page 2, line 48)

Author Response

Kindly see file.

Author Response File: Author Response.docx

Reviewer 2 Report

The paper addresses a current and important topic from the perspective of UX - Covid 19. The described approach, the related work and also the applied methods are in line with the state of the art of scientific research. The methods used for statistical analysis and the presentation of the results are also in line with scientific standards. However, one criticism of the work is that the evaluation of the concept is based on questionnaires and not on interaction studies (which ideally take place over a longer period of time), because the longer-term effect of an interaction concept, e.g., frequency of use, or the central aspect of the work - persuasiveness - can, in my view, be better evaluated with this kind of method. In this context, I would also miss a basic literature reference, namely B.J. Fogg - Persuasive Technology.

The paper has some minor typos: 

Page 2: Related work: line 77  - They found (instead of the found), Page 3: Method - line 104: persuasive features (instead of persuaisve), page 6: Research model - line 169: perceived (percived), aesthetics (aestehtics), Page 7: evaluation of the measurement models: line 196: two times "the",  adopter (adoopter), in line 198 and 200 there seem to be left some placeholders (X) - please replace with data.    

Author Response

Kindly see file.

Author Response File: Author Response.docx

Round 2

Reviewer 1 Report

The paper has been sufficiently improved to be eligible for publication in MTI. Minor remarks:

- new Figure X should be numbered (and other figures consequently re-numbered)

- "Study objective" (lines 173-180) would be better positioned before "Overview of COVID Alert"

- line 400: percived -> perceived

- line 479: "We hope to examine .." -> "For example, we plan to examine .. "

Author Response

- new Figure X should be numbered (and other figures consequently re-numbered)

Response:

It has been fixed.

- "Study objective" (lines 173-180) would be better positioned before "Overview of COVID Alert"

Response:

It has been repositioned.

- line 400: percived -> perceived

Response:

It has been corrected.

- line 479: "We hope to examine .." -> "For example, we plan to examine .. "

Response:

It has been corrected.

Appreciation:

Thank you very much for taking the time and effort to review our paper.

Author Response File: Author Response.docx