A Study on the Psychology of Social Engineering-Based Cyberattacks and Existing Countermeasures

Round 1

Reviewer 1 Report

1) In introduction, introduce the problem, motivate the problem, and summarize the main contributions in detail

2) Authors are suggested to add related papers on deep learning with Social Engineering-based Cyberattacks works. Some are given below

https://link.springer.com/chapter/10.1007/978-981-15-4828-4_14

https://ieeexplore.ieee.org/abstract/document/8854532

https://www.riverpublishers.com/journal_read_html_article.php?j=JCSM/8/2/3

3) Authors can also add a section to discuss about the avaialbe datasets for deep learning with Social Engineering-based Cyberattacks works

4) Authors can discuss about the limitations of exsisting deep learning with Social Engineering-based Cyberattacks works and its future works

5) Authors can discuss the missing information from the exisisting related survey papers and how the proposed work is different comapred to the exsisting methods in the introduction

Author Response

Please see the attachment

Author Response File: Author Response.docx

Reviewer 2 Report

I think there is a good intuition that Social Engineering exploits fundamental psychological aspects/properties/processes that are fairly common to everyone, so it's interesting that it has taken so long to get a good overview!

The topic and problem space is well justified, both by the authors, and by the reality of the world. I actually didn't need much convincing here as I have recently supervised an MSc thesis which had the same (good) idea.

In terms of contribution, the paper seems to be achieving two main things:

- An overview of the space, including psychological processes, SE landscape, and potential solutions

- Identification of key psychological principles and how they apply to SE, with some examples which are expanded upon to make the point

Generally I think both of these are achieved, but I have two major comments:

- The literature for psychology does not appear to be cited properly in the body of the document, even when papers are named directly (e.g. Cialdini on line 185). This means there is quite a large chunk of the literature that appears in the references section without ever being cited.
This is an important oversight, particularly as the foundation of the document relies on linking these works in psychology (a domain in which computing researchers are likely lacking in expertise, so they are much needed) to the computing context. This needs to be done robustly, and it currently isn't.

- I feel that too much of the paper focuses on the wrong kind of technical solution analysis. I understand that the authors are making an argument for technological defenses not being enough, but this appears to be dwelled upon too much in the latter part of the document. By the time I reached page 11 I did not want to read about NLP for detecting malicious URLs, I wanted to know what chance we had of applying NLP to identify the psychological techniques that were identified in Section 3. The authors touch on this idea in passing, but it really should have been the main thrust of the discussion in Section 4 in a much more direct fashion. I'm not suggesting that the authors create/develop solutions, but rather provide a cursory identification of existing techniques and their limitations in the SE/psychology application domain. (e.g. from the realms of NLP, argumentation processing, etc.). This would help fit the overview contribution and give anyone who wants to work on technical mitigations a head start.

On a lesser note, I found it curious that there doesn't seem to be any acknowledgement of the relatively small body of work which has already done this kind of thing, albeit at a lesser scale. For example:

Kabay, M., Robertson, B., Akella, M. and Lang, D., 2015. Using Social Psychology to Implement Security Policies. Computer Security Handbook, pp.50.1-50.25.

Sibrian, J., 2021. Sensitive Data? Now That's a Catch! The Psychology of Phishing. Bachelors. Harvard College.

In terms of fitting with the venue, it does seem like the AI/ML slant in the paper is largely to better fit the category, but that could just be my perceived bias as a non-ML researcher. Sub-sets of AI/ML are likely going to be the solution to identifying many of these psychological flags, at least where that is possible, so it is a reasonable perspective to take.

Overall my inclination is that this work needs to be published, by someone, somewhere, as there is a gap in the literature begging to be filled.

This work does a good job of identifying all of the major components, though it definitely needs to address the citation issues mentioned above for this to work properly.

As such, my recommendation is to accept the paper upon revising the citation problems. I feel that the paper would be much more useful/improved by more directly discussing the findings of Section 3 in the latter parts of the paper, but this is not something that I would require of the revision. That being said, I do encourage the authors to consider it!

Author Response

Please see the attachment

Author Response File: Author Response.docx