Instruments and Tools to Identify Radical Textual Content

Round 1

Reviewer 1 Report

In this paper, authors presented a set of instruments and tools developed within the PREVISION project. However, there are some limitations that must be addressed as follows.

1. In abstract, some sentences look a kind of repetition. For example, “This paper presents…” and “the tool we present in this paper…”. These sentences should be modified in order to clearly represent the novelty of this work.
2. In Introduction section, the main novelty is not clear. The main contribution should be presented in the form of bullets.
3. How about related works? the authors should include a separate section and discuss the existing work. In addition, the authors should also discuss that how the twitter and Facebook data is handled in the existing work. Discuss the following existing work (‘A Review on the Detection of Offensive Content in Social Media Platforms’ Traffic accident detection and condition analysis based on social networking data’, Big Data Analysis of Facebook Users Personality Recognition using Map Reduce Back Propagation Neural Networks).
4. Captions of the Figures and tables not self-explanatory. These captions should be self-explanatory, and clearly explaining the figure. Extend the description of the mentioned figures and tables to make them self-explanatory.
5. The key phrases in Section 4.2 should be more clearly represented, better to include arrow (->).
6. Figure 5, 8, 10 is blurred, difficult to read.
7. More detail should be included in Conclusions.
8. The future work should be discussed at the of the conclusion.

Author Response

We would like to thank the reviewers for their valuable comments that helped us to improve the quality of this submission.

We have read carefully the reviewers' comments and have taken them into account.

(1) Repetitive sentences in the abstract have been removed, as well as in the introduction.

(2) The main originality of the tools are now presented in the introduction

(3) We have added a related work section

(4) Figure and Table captions have been revised

(5) Figures have been updated to improve readibility

(6) we have complemented the conclusion and added future work

(7) Moreover, we have completely restructured the submission. We first present the contribution and tools in a separate section; then the application to the use case (radicalisation) is presented in a separate section. We think that makes the contribution clearer. We also reshape the discussion and conclusion section.

Author Response File: Author Response.pdf

Reviewer 2 Report

The paper presents the instrument and tools for the identification of radical text published online (social media, Internet). The paper presents the extensive tools used for the detection of radical language that is combined together in the application for calculating the radicalization score of text files for a weighted lexicon. The paper is written from the perspective of one EU project, which makes it hard to follow it and to assess the value of the results. 

1. The paper should be rewritten so that the methodologies for the identification of radical content are presented first. After that, the usage of these tools in the EU project should be presented as a case study.
2. The abstract should be rewritten so that the focus is the tools for the identification of radical content, and that the EU project is mentioned as the case study for using these tools.
3. Authors should take care on their language, since they are also politically incorrect, e.g. by mentioning "extremists of all colours". 
4. The introduction section needs to be rewritten with much better motivation and provide the context for this work. It should include: (1).     Contextualization (2).     Importance/Relevance of the Theme (3).     Research Question (4).     Objectives (5).     Structure of the Paper
5. The phrase "toy use case" should be replaced with the expression "example use case". 
6. The new chapter Overview of instruments and tools for identification of radical content should be formed, and in this chapter, the methodologies should be described. For each of the tools, there should be a sub-chapter. 
7. The new chapter Methodology should be included, which would consist of the subchapters Data and Methods. In this chapter, the EU project should be described. 
8. The new chapter presenting only results of the application of the tools on the use case should be formed and called Results. It should be structured with the same subchapters, as the chapter Overview of instruments and tools for identification of radical content.
9. The chapter Ethics and data protection should be renamed into Discussion, implication, and conclusion.  In the last section, please focus on “Discussion, Implication, and Conclusion” to include (1).     Summary of the research - what was the goal and how it was attained (2).     Discussion why the authors found out these results and how they comply (or not) with the Literature Review? (3).     Managerial Implications (4).     Limitations of the paper (5).     Future Studies and Recommendations

The notion of terrorism should be discussed also from the perspective of economic impact, e.g. Pejić Bach, M., Dumičić, K., Jaković, B., Nikolić, H., & Žmuk, B. (2018). Exploring impact of the economic cost of violence on internationalization: Cluster analysis approach. International journal of engineering business management, 10, 1847979018771244.

In the end, I would like to encourage the authors to re-write their paper, since I believe that it presents highly-valuable results, but which are hard to follow in the present form, where the paper is written from the perspective of the project, while it should be written from the perspective of the problem and results, using the project as the example. 

Author Response

We would like to thank the reviewers for their valuable comments that helped us to improve the quality of this submission.

We have read carefully the reviewers' comments and have taken them into account.

(1) we followed the reviewer's recommendation so that now the contributions are first presented in a general way, then they are applied to the project use case radicalisation.

(2) Abstract has been rewritten

(3) "extremists of all colours" which was not appropriate has been removed. We did not find other phrases that may offend people or were not appropriate.

(4) We rewrote the introduction and make clearer the context, objectives, contribution and structure of the paper.

(5) "toy use case" as been replaced with the expression "example use case". 

(6) We followed the reviewer's recommendation and added a new chapter that provides the Overview of  the tools; with subsections, one for each of the methodologies/tools.

(7) We added the Methodology section as suggested, including Data and Methods, where we describe the EU project.

(8) We added a new chapter centered in the results of the application of the tools on the use case, as suggested.

(9) Finally, the Ethics and data protection section has been included in the Conclusion section. That section has been completed, so that it includes the summary of the contribution and the future work as well.

(Other): we added a related work section

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

The authors have not addressed my comments properly. All the comments should be addressed properly. 

Author Response

Dear reviewer,

We though we have answered the reviewer's comment. However, the reviewer on Round 2 wrote "The authors have not addressed my comments properly. All the comments should be addressed properly."

We though went back to his or her comments on round 1 and answer here point per point. We hope this answers the reviewer's comments.

Round1

1. In abstract, some sentences look a kind of repetition. For example, “This paper presents…” and “the tool we present in this paper…”. These sentences should be modified in order to clearly represent the novelty of this work.

In the revised version, the repetitions have bee removed we think.

The Internet and social networks are increasingly becoming a media of extremist propaganda. On homepages, in forums or chats, extremists spread their  ideologies and world views which are often contrary to the basic liberal democratic values of the European Union. It is not uncommon that violence is used against those of different faiths, those who think differently, and members of social minorities. This paper presents a set of instruments and tools developed to help  investigators to better address hybrid security threats, i.e. threats that combine physical and cyber attacks. These tools have been designed and developed to support security authorities in identifying extremist propaganda on the Internet and classifying it in terms of its degree of danger. This concerns both extremist content on freely accessible Internet pages and content in closed chats. We illustrate the functionalities of the tools through an example related to radicalisation detection; the data used here are just a few tweets, emails propaganda, and darknet posts. This work was supported by the EU granted PREVISION
(Prediction and Visual Intelligence for Security Intelligence) project.

1. In Introduction section, the main novelty is not clear. The main contribution should be presented in the form of bullets.

In the introduction bullets were added in the revised version:

The main originality of the tools are as follows:
* The first series of tools we developed relies on a lexicon built thanks to keyphrase extraction. Recent related work has developed new algorithms for keyphrase extraction~\cite{mothe2018,campos2020yake,rose2010automatic}. In the literature, these keyphrases are used for information retrieval purposes. Here, we develop an original use of the keyphrases which is to rank documents, not according to a query, but rather according to the lexicon itself. Because the lexicon is representative of a sub-domain a LEA is interested on (e.g. religious extremism), it is the possible to order the texts according to their inner interest for the user; 
* Document ordering is based on two means. One of them relies on the expert knowledge of the importance of some criterion. In this, our models are more task oriented than the usual models;
* Current search engine does not explain the user the results that it retrieved. As opposed to that, here, the link between the lexicon and the text is highlighted so that the user can understand the reason of the document ordering (and can agree/disagree with the results);
* Visual tools complement the underlying representation of texts and help the user understanding what the texts are about by an overview of the main important terms;
* One fundamental point of PREVISION is that it integrates all the elementary tools into a complete processing chain which is directly usable by LEAs; such platforms does not exist where the user keeps the control of the system results;

1. How about related works? the authors should include a separate section and discuss the existing work. In addition, the authors should also discuss that how the twitter and Facebook data is handled in the existing work. Discuss the following existing work (‘A Review on the Detection of Offensive Content in Social Media Platforms’ Traffic accident detection and condition analysis based on social networking data’, Big Data Analysis of Facebook Users Personality Recognition using Map Reduce Back Propagation Neural Networks).

We added Section 2. Related work section

1. Captions of the Figures and tables not self-explanatory. These captions should be self-explanatory, and clearly explaining the figure. Extend the description of the mentioned figures and tables to make them self-explanatory.

Table and Figure captions were completed in order to make them self-explanatory.

e.g. Figure 1 caption is now "Text Visualisation Tool (TVT) workflow. The user opens TVT tool in a web user interface where he can provide the text to be analysed. One can also modify the text visualisation parameters (e.g., number of expected topics) or keep the default values. Finally (after the user clicks ``generate") the visualisations are generated and displayed in the web user interface.

1. The key phrases in Section 4.2 should be more clearly represented, better to include arrow (->).

This was improved (see page 13) Now section 5.3

1. Figure 5, 8, 10 is blurred, difficult to read.

Figures were changed. Moreover, Figure 10 was redone because it was to small to be readable proprely

1. More detail should be included in Conclusions.

We had enlarged the conclusion to summarize the paper content , discuss and conclude it

1. The future work should be discussed at the of the conclusion.

It was done

Author Response File: Author Response.pdf

Reviewer 2 Report

Dear authors, the paper is now acceptable for publication. 

Author Response

Many thanks for your reviews.

We have proof-read again the paper and managed to correct some more errors.

Best regards