Developing Analytical Tools for Arabic Sentiment Analysis of COVID-19 Data
Round 1
Reviewer 1 Report
The paper is dedicated to such important topic as sentiment analysis. Also I see importance of analysis Arabic language text which can widen the view on different languages on topic. As positive I also appreciate that you provided enough image and table supplement material. In overall, I don’t see any errors in the methodology or the experiment, so we kindly ask to correct formatting according the below guidelines.
I have some questions to your work, related to formatting. I not sure whether to treat them all as issues or suggestions, but due to the number of them, I want to be sure the vast majority of them to be fixed.
1. In keywords section it is better to utilize such keywords: Arabic datasets, sentiment analysis, Twitter, emoticons, COVID-19. They underscore the importance of area of interest and to widen the audience.
2. Some formulae have formatting issues:
- Formula 1 and 2 use upper case for items inside terms like: NumberOfTweets to improve readability.
- Formulae 3 and 7 should have other sign for dot product to not to be confused with the decimal point.
3. Some tables have formatting issues:
- Table 4 titles overlap
- Tables 5 and 9 not fit the width
4. Text issues:
- Line 308 text not fit the width
- Line 158 no punctuation mark
5. Figure issues:
- Figure6 is misplaced and not fit the width
6. Citation issues:
-Line 104, 127-133, 154, 156, 393, usage of citation in the beginning of the sentence
-26 sources out of 45 are more than 5 years old.
For my our opinion issues 2.2, 3.1, 3.2, 4.1, 4.2, 5.1, 6.1 are critical and have to be fixed. Other ones, like 1, 2.1 and 6.2 are not critical and left upon authors’ consideration. Also I want to raise the attention to placement of the images and tables WRT their first appearance in the text. This is specially seen with issue 5.1 as terminal example.
Author Response
Dear Reviewer,
Thank you for your valuable input and feedback. We found your comments extremely helpful. We look forward to hearing from you regarding our submission and to responding to any further questions and comments you may have.
Author Response File: 
Author Response.docx
Reviewer 2 Report
General: affiliation data is not provided.
l.5-6 Is it two-class or three-class sentiment analysis task? It is unclear from the introduction.
l.10 The definition of "lexicon accuracy" is not clear.
l.14 What are the four feature groups?
l.61-62 Please explain how many emotion categories you consider. Are these the 5 standard classes, or more fine-grained type of emotion recognition?
l.63-64 Please provide more details about Arabic SA corpora that are available. A good place to start would be a survey paper
Alsayat, A. and Elmitwally, N., 2020. A comprehensive study for Arabic sentiment analysis (challenges and applications). Egyptian Informatics Journal, 21(1), pp.7-12.
that contains a list of corpora in Table 2.
l.76 There is an Arabic covid-related corpus available, please see:
Aljabri, M., Chrouf, S.M.B., Alzahrani, N.A., Alghamdi, L., Alfehaid, R., Alqarawi, R., Alhuthayfi, J. and Alduhailan, N., 2021. Sentiment analysis of Arabic tweets regarding distance learning in Saudi Arabia during the COVID-19 pandemic. Sensors, 21(16), p.5431.
Please explain what are the differences and where your new corpus is significantly different.
l.196-197 Please provide an example of an opinion - it is not quite clear.
l.212-224 A table summarizing these decision would be helpful, because this description seems incomplete.
l.233 'Joking' is not a sentiment according to the SA task definition. Please provide references that justify giving it its own class. In general, such texts would be attributed to irony or sarcasm, and their detection are separate tasks that cannot be called SA or emotion recognition.
Also, ignoring sarcasm and irony makes the whole task much easier because these traits are the hardest to detect in texts.
l.245-250 Please make these descriptions into equations, they will be much easier to read.
l.279 How was the manual testing conducted? How many annotators were there and what is their education/gender/background/mother tongue? What was the agreement between them?
l.329 Tokenization is non-trivial in Arabic, please explain what method/pretrained model and what implementation did you use?
l.348 What are those opinions related to? For opinion detection, one needs a target in order to decide whether or not a text is 'pro' or 'con' or 'neutral'.
l.369 Again, for these three classes, you are speaking of sentiment orientation and not the emotional one. Emotions need at least five classes.
Section 4.1.2 It is more of a representation than feature extraction.
p.15 eq.(1)-(2) need to be fixed. They are ill-formatted and in case of an IDF, contain a mistake.
l.421 Again, these are sentiments and not polarities. Polarity usually is a numeric weight of a sentiment, in which case you need to specify a range.
l.480-492 There is no need to repeat the scores here as they are provided in Table 9.Table 10: Your results need to be compared to some baselines. There is an Arabic SA SW package available at https://github.com/motazsaad/arabic-sentiment-analysis, please run it.
Also, please use AraBERT transformer and fine-tune it on the training portion of your data:
Antoun, W., Baly, F. and Hajj, H., 2020. Arabert: Transformer-based model for arabic language understanding. arXiv 2020. arXiv preprint arXiv:2003.00104.
Alas, train/test division of data is not mentioned.
p.18-19 The same issue as above, numeric results given in these tables should not be repeated in paper text.
Some minor typos.
Author Response
Dear Reviewer,
Thank you for your valuable input and feedback. We found your comments extremely helpful. We look forward to hearing from you regarding our submission and to responding to any further questions and comments you may have.
Author Response File: Author Response.docx
Round 2
Reviewer 1 Report
Thank you for accepting my remarks. So I think the paper my be published.
Author Response
We would like to thank you for taking the necessary time and effort to review the manuscript. We sincerely appreciate all your valuable comments and suggestions, which helped us in improving the quality of the manuscript.
Reviewer 2 Report
Most of my comments were adequately addressed. However, I did not find results for baselines in page 18 as indicated in the authors' response letter. Please add these results to the paper.
Author Response
Dear Reviewer,
We would like to thank you for taking the necessary time and effort to review the manuscript. We sincerely appreciate all your valuable comments and suggestions, which helped us in improving the quality of the manuscript.
Point 1 : Most of my comments were adequately addressed. However, I did not find results for baselines in page 18 as indicated in the authors' response letter. Please add these results to the paper.
Response 1:
Thank you for your suggestion. We have incorporated more clarification accordingly in the updated version of the manuscript. This part is updated in the manuscript on page 18.
