An Abstractive Summarization Model Based on Joint-Attention Mechanism and a Priori Knowledge
Round 1
Reviewer 1 Report
This paper presents an approach towards automatic summary creation that is claimed to outperform other well-known algorithms.
I will stand into two basic issues related to the paper, before presenting detailed information and comments. The first one is related to the language. The paper has so many language flaws that it is very difficult to read and understand. The second and most important is the fact that the system is based on algorithms considered to be “dated” after the presentation of the novel AI algorithms and especially transformer machine learning model.
Going back to analysis of the paper, the abstract is not representative of the paper. It seems like random sentences were selected and put in an order, but it does not make sense and does not represent the work that was done.
Lines 27-28 extractive and abstractive summaries are mentioned, only the first one has reference, the second one not.
Line 31 compared TO
Line 33 ..it is complicated to implement.. does not sound like a probationary term/sentence
Line 33-34 “abstractive abstracts”, while it was earlier mentioned abstractive summaries…
Line 39 mentions computer vision as a field that was related to deep learning, but it is irrelevant to this work.
Line 40-41 there is a need for a reference
Line 42, “which is presented in figure….”
Line 42: “the most popular one”…. Among what things?
Line 47: proposed AN abstractive
Line 49: “And then…”. This start of sentence is not suitable
Line 55-56 “has achieved good results”: this does not prove efficiency or effectiveness
Line 56: OOV problem is mentioned without ever saying what is the oov problem
Line 57: Generator – Pointer model is mentioned but not explained
Line 58: Exposure Bias is mentioned but not explained
Line 61: mentions “cross-entropy loss loss”. Correct is cross-entropy loss. Error is mentioned twice in the text
Line 62: Other work is mentioned but not referenced
Line 72-80: Syntax errors in sentences make them difficult to understand
Line 91: “it is deciding”. Wrong syntax
Line 92: be attended. Wrong syntax
Line 97-99: syntax error
Line 102: DAS is mentioned but not explained
Lines 107-108: syntax error
Line 110: “cascading or in parallel” – wrong syntax
Line 114: key and value PAIRS
Line 125 cross-entropy loss loss error
Line 128-131: syntax error
Line 136: Kryscinski – error in name
Line 144: only the more important => only the MOST important
Line 147: Information in the original text. There is a strong need to define what is information
Figure 3: Priori Knowledge => A Priori Knowledge? OR generally what they want to mention is prior knowledge. It is important to make it clear if “a priori” is meant or “prior”.
Line 168: infinitely close ? does not have soundness
Line 200-201 “gradient from disappearing”?? does not have soundness
Line 204: “…to guide the generation…” syntax error
Line 209 – 211: incomprehensible
Equations 9 – 16 are present but there is no detailed explanation per equation in order to realize how the algorithm is executed.
Line 234 “that is to say…” = not so formal
Line 256: “We found that humans….”: this is an assumption that is not substantiated
Eq 19 is the average of weights. How then Sm is considered to be a “higher level semantic representation”??
Line 280 presents the recurring problem to be: “I like you like you like you…”. This is an extreme case of algorithm error and not the recurring problem of summarization algorithms.
Equation 22 is just a summary and not what is claimed to be in the text.
Line 299: “we propose solutions to the above problems” should be expressed more precisely.
Line 301: “by an intelligence” – error
Line 305 – 307: syntax error
Line 317: “use the ROUGE score as the reward function”. This does not stand as a sentence! Moreover you need to decide how to mention ROUGE. In the text it is mentioned as Rouge, ROUGE and rouge. Please have one default term.
Line 347: “this paper studies…”. It seems that the paper does not study, rather, “the research utilizes” or “the research performs tests on”…
Line 352-353: syntax error
Line 367: “rate in Eq ()” – which eq?
Line 371: “As shown in Eq ()” – which eq?
Line 373: syntax error
Line 375: “we manually selectED”
Line 377: there is a strong need to have information about the evaluators as well as the exact dataset and their questions. When it comes to manual evaluation more evidence is needed.
Line 412: Transformer does not have a reference.
Table 1 needs more clarification and more information on the data used. Other research present that Transformer is much better that other algorithms, while this paper presents Transformer to be slightly better.
Table 2 is subjective and as such the complete set of data and the answers need to be provided.
Table 3 which includes the tests on DUC performs comparison with algorithms that are 4-7 years old. We live an era that these kind of algorithms keep changing day by day
Line 455 – 458: incomprehensible
After all the aforementioned, it is quite clear for me that the test result does not have soundness and need to be presented from different angles while subjective data need to be presented.
Author Response
We appreciate your thorough review. We addressed each of your suggestions separately. Please see the attachment.
Author Response File: 
Author Response.pdf
Reviewer 2 Report
This research paper proposes a new abstractive summarization model based on joint attention mechanism and infusing prior knowledge.
The methodology and model architecture are well-explained with formulas.
The result of the model are also compared to some of the advanced models and datasets publicly available.
Even though the performance of the model is on-par with or even lower in some cases compared to other models, the small advantages it could gain in performance are worthy of discussion and interesting for future references.
The authors have proposed a new neural architecture to generate abstractive summaries from text documents using word and sentence-level decoding and infusing prior knowledge with pointer networks.
Author Response
The article has numerous grammatical flaws as a result of our blunders. As a result, the article is difficult to read. We sincerely regret that this has inconvenienced you. Thank you for your patient reading and detailed review.
Due to the high requirements for servers and excessive time costs of pre training models, there is a lack of research on pre training models. The evaluation score of the summary generated by the model in this article is slightly lower than the result of the pre training model. In the future, we will conduct in-depth research on the pre training model to determine whether the introduction of the pre training model will further improve the quality of the summary model.
Reviewer 3 Report
1-The introduction does not provide motivation of this work (importance of text summarization)
2- The problem statement is not well defined in the introduction
3-From line 42 to line 66 could be splited in a background section
4- A figure to describe the overall process is required
5- The example shown in figures (4,5,6) respectively required real data
6- English language revision is required for the following statements :
line 129-131, 373 - 375
Author Response
Thank you for taking the time to review the paper. We addressed each of your suggestions separately. Please see the attachment.
Author Response File: Author Response.pdf
Reviewer 4 Report
The focus of the paper is clearly defined and the proposed architecture gives better results compared to other models.
However one issue is not completely clear to me in the approach: the one related to the keywords. How did you extract keywords from texts? This comment is related to the claim presented in the paper at lines 192-193, 203-204, 498-499.
From the presentation perspective, the whole paper needs to be revised by an English native speaker. There are some sentences very difficult to understand. To give some examples:
- Abstract, line 3-4: First filter the word --> this is the imperative construction, please revise.
- line 6: After this processing --> which one? please clarify
- lines 16-17: the 1st sentence is not an English construction. My suggestion is: "Before the rapid development of the Internet and common electronic devises, people got news and other information from newspaper and TV. Nowadays, network platform ... "
- line 28: According to the output type, it .. --> "it" is referred to what?
- line 49: And then --> not a formal expression, too colloquial
- et al --> et al. (throughout the paper)
- line 90: The attention mechanism is the idea .. --> "An attention mechanism meets the idea .."
- line 141: In this paper, the source .. --> "In this paper, we present a model where .."
- line 230: nth --> th should be as superscript.
- line 373: I cannot understand it... "manual evaluation new mainly evaluate .." what does it mean?
Author Response
Thank you for taking the time to review the paper. We addressed each of your suggestions separately. Please see the attachment.
Author Response File: Author Response.pdf
