An Easy Partition Approach for Joint Entity and Relation Extraction

Round 1

Reviewer 1 Report

This article aims to extract triples from the fusion of NER and RC. They provide a Pre-trained Language model based on tokens and fusion steps to improve performance. 

Providing examples concerning their approach is important to ease the reader's understanding. Some parts of the text are hard to follow the authors' ideas. What do the authors mean when they describe semantic feature generation? What is semantics in such a context? I suggest discussing it more. 

 

There is an explanation of their approach concerning table filling. What do authors mean by Table filling? I suggest discussing it more. 

 

What is partition semantic representation information? It needs more discussion on that. 

 

Where is the feature generation module in Figure 1? It is not clear in the manuscript. 

Equation 4 lacks a ], or all others. As equations g and g', is it possible to not have a triple without h_ge or h_gr? Only with h_share ? 

 

What do you mean by "merge features'? 

 ADE has only two entity types. I suggest more discussion on that. 

The authors stated that they provide two different types of encoders. I suggest discussing and giving more information. Do authors think this may bias some results?

Some variables were not explained in the evaluation aspects, and the results were not discussed. What do you mean by 5019 years? 

The authors stated that some errors were propagated from the dataset. It is important to describe which are they and their percentage and influence on the method. 

Concerning the new concept of density, I suggest providing an example. 

 

I miss some quality extraction evaluations. How is the performance of complex sentences with complex relations and entities? 

No linguistic description of the dataset was provided. 

 

 

Considering these comments above, results need to be organized and improved. 

 

References need to be formatted. 

 

 

Some typos

-> tokes -> tokens

four publicly -> three publicly 

PFN - FPN

Author Response

Thank you for your comments concerning our manuscript entitled "An Easy Partition Approach for joint Entity and Relation Extraction" (ID: applsci-2294575). Those comments are all valuable and very helpful for revising and improving our paper, as well as the important guiding significance to our research. We have studied the comments carefully and have made a correction which we hope meet with approval. Revised portions are marked on the paper.  

The main corrections in the paper and the responses to the reviewer's comments are as follows:

 

1. Thank you very much for your frank comments. In our paper, we used the token embedding obtained from the pre_trained language model (PLM) as the mathematical representation of semantic representation information. In response to your suggestion, we revise our manuscript and provide a clearer example in this fused module. We hope that these changes will make our paper easier to follow and understand for our readers.
2. Thank you for taking the time to review our paper and provide us with valuable feedback. We appreciate your insightful comments and would like to address your concern regarding our approach to table filling. As we mentioned in the related work section of our paper, we introduced the table filling method for the first time. In section 3.2.3 of our paper, we provided a detailed explanation of the specific operations involved in table filling. We agree that discussing this approach further would improve the completeness of our paper. Once again, we sincerely appreciate your feedback, and we hope that our response adequately addresses your concerns.
3. In this paper, we get the token embedding obtained from PLM as the mathematical representation of semantic representation information. Then, based on the required number of features for the task, we divided the embedding into three parts using the chunk function. We have explained this process in our paper with an example: "If the embedding dimension of one token is d, then we split it into three parts according to three tasks, and each part's dimension is d/3. The features of the three parts are defined as the shared feature representation "h" _"share" , NER feature representation "h" _"ner" , RC feature representation "h" _"rel" ". Additionally, we have addressed your concern by using a sentence ("The current CEO of Apple is Cook") to provide a better explanation of the entire model operation in the fused module section of the paper. Once again, we appreciate your valuable feedback and suggestions.
4. We have replaced the term "feature generation module" with "encoder module" in the manuscript to improve clarity and aid reader comprehension. We have also created a new version of Figure 1 based on the model architecture, using Visio to ensure a high-quality image, and included it in the manuscript. We appreciate your valuable feedback and input to help us improve the manuscript.
5. We have made the necessary modification to equation 4 in the manuscript. Regarding your question on equations g and g', it is possible to have a triple without h_ge or h_gr, and only with h_share. And, we have implemented and compared this approach with the one presented in the manuscript in the ablation study section, and found that the model's performance is better when using the proposed approach. We suspect this is because our approach takes into account both the uniqueness and commonalities of the tasks. Thank you again for your valuable feedback.
6. To provide further clarification, when we mention "merge features," we are referring to the process of combining task-specific features and task-share features to create the final features used for the NER and RE tasks. This process involves weighted feature concatenation, as shown in equations 4 and 6, which employs a trainable gating mechanism to determine the feature weights. In order to help readers better understand our model, we have included an example in the fused module section of the paper to illustrate how the model operates.
7. As you mentioned, the ADE dataset only has two entity types, which we agree with, and this is one of the reasons why we introduced the dataset in the paper. The ADE dataset is specifically designed for the task of entity relation extraction in medical texts, and it includes two entity types: Drug and Adverse Drug Event. These two entity types are crucial in the medical field, with Drug referring to medications and Adverse Drug Event referring to adverse events related to drugs. Therefore, they are essential for entity relation extraction tasks in the medical domain. Although the ADE dataset only contains two entity types, it has a large amount of data and is highly representative in the medical field. Thus, it is one of the most important datasets for studying entity relation extraction tasks in the medical domain. We have provided a detailed description of the ADE dataset in the paper, hoping to provide readers with a deeper understanding of the dataset.
8. Thank you for your suggestion. In order to capture domain-specific information, we used different pre-trained language models (PLMs) as encoders for different datasets. We believe that this approach can improve the performance of our model on diverse datasets. However, we acknowledge that this may introduce some bias in our results. To ensure fair comparison, we explicitly reported the encoder types of our model in Table 3 and compared our results with other models that used the same encoder on the same datasets. We hope that this provides sufficient transparency and helps readers to evaluate the performance of our model.
9. In the "Main Results" section of our paper, we discuss the performance of our model on each dataset and analyze the main types of errors produced. We also conducted ablation research in the next section to evaluate the effectiveness of our model's three modules and fully discuss the performance of our model under different variant conditions. Thank you for pointing out the error regarding "5019 years." What we meant to convey was that the size of the training dataset for webNLG may have reached a saturation point for the model. We have corrected this error in our manuscript.
10. We have investigated the reasons behind the errors generated by our model, starting from the dataset itself. Insufficient manual annotation of the dataset is one of the factors, which has also been mentioned in the TPLinker paper (TPLinker: Single-stage Joint Extraction of Entities and Relations Through Token Pair Linking). However, due to time constraints, we were not able to conduct a comprehensive evaluation of the quality of the entire dataset. We have performed experimental validation and evaluation of our paper's performance in terms of entities. We have found that our model's performance is poor for short entities (entities with only one word), and we have provided specific entity-level results. The experiment results is shown in the table. We also observe that EPRE and PFN on Out_triple F1 values (46.5%, 48.6%) were significantly lower than in In_triple (75.4%, 76.2%). We believe that this phenomenon is due to the low cohesion between short entities and other words, as well as the limited contextual information available for out_triple entities.

	Our		PFN
Data_length	NER	RE	NER	RE
long	p=0.6842, r=0.6802, f=0.6822	p=0.4181, r=0.2797, f=0.3352	p=0.6746, r=0.6836, f=0.6790	p=0.3727, r=0.3166, f=0.3424
Medium	p=0.6798, r=0.6434, f=0.6611	p=0.4531, r=0.2802, f=0.3463	p=0.6995, r=0.6783, f=0.6888	p=0.3720, r=0.2947, f=0.3288
Short	p=0.5600, r=0.5385, f=0.5490	p=0.2500, r=0.1868, f=0.2138	p=0.5686, r=0.5577, f=0.5631	p=0.2500, r=0.2222, f=0.2353

 11. We appreciate your suggestion, and we agree that providing an example is a helpful way to explain the concept of density. In our paper, we have added an example to explain density as follows: "For example, in the sentence "The current CEO of Apple is Cook", we can observe that there are three entities (CEO, Apple, Cook) and the number of words in the sentence is 7, so the density of the sentence is 3/7." We hope that this addition will make our paper more complete and easier for readers to understand.

12. We followed the approach of previous works such as Casrel and TPlinker in evaluating the quality extraction performance for complex relations and entities. We believe that the more triples a sentence has, the more difficult it is for the model (TPLinker: Single-stage Joint Extraction of Entities and Relations Through Token Pair Linking; A Novel Cascade Binary Tagging Framework for Relational Triple Extraction). Additionally, sentences with Entity Pair Overlapping (EPO) triples are more challenging compared to sentences with Single Entity Overlapping (SEO) or normal triples. We reported our results on these aspects in section 4.2.3 of our paper.

13. We have added a description of the dataset in Section 4.1.

14. We have revised and formatted the references according to the journal requirements. If you have any further suggestions, please let us know.

15. It was our mistake. We have carefully reviewed and revised the typos you pointed out in our manuscript.

Thanks again for your input. We truly appreciate your time and suggestions!

Author Response File: Author Response.docx

Reviewer 2 Report

Overall, the model for triple extraction proposed by the authors achieved credible performance on public datasets. In addition, the innovative approach proposed in this study has academic significance. 

However, the introduction section is too brief and should be expanded to enhance reader's interest. It may be helpful to use a table format to present related research. 

I believe that, with such the following revisions, this paper could be considered for acceptance:

1. Please provide more information about the performance of the model during the training phase.

2. Consider redrawing Figure 1 or adding a more detailed description to clearly show the three modules mentioned by the authors.

3. The authors compared the computational efficiency in the results, but they do not mention the hardware platform used. Please add a note about the hardware platform.

4. Increase the resolution of the image much more.

 

The problem addressed in the paper is that the existing methods are unefficient to extract triplets from a given text. So, the authors proposed an approach employing pre-training language models and feature partition operations to obtain task-specific features with better accuracy and efficiency. This method is simpler and faster than existing approaches and has demonstrated state-of-the-art performance on benchmark datasets for NER and RC tasks. The comparison with the state-of-the-art methods on public datasets provides strong evidence for the effectiveness of the proposed method.

 

However, while there are provided an ample evaluation and discussion of the findings, it is very hard to find what makes the difference. The method described in Section 3 is not sufficient to figure out how the feature partitioning method and feature fusion strategy is working and how the gating mechanism is applied. The authors just presented the idea  with a mathematical form again, which is not helpful to understand the overall mechanism. A more detailed explanation using a concrete example is definitely needed.

 

In addition, a proofreading is required:

For example, in abstract, 'neither joint or separate encoding' should be 'neither joint nor separate encoding'. In page 4, the sentence does not make sense: "We take the pointer network to predict the segment from i to j is whether an entity of entity type k."

Author Response

Thank you so much for your kind words and valuable feedback. We are grateful for your recognition of our work and have taken your suggestions seriously. The introduction section has been revised to make it more appealing to readers, and we have implemented your excellent suggestion of using a table format to present related research in the paper. This has certainly enhanced the overall presentation of the previous studies in the related work section. The main corrections in the paper and the responses to the comments are as follows:

 

1. Following your suggestion, we have included more information about our model's performance during the training phase. Specifically, we have provided details on precision, recall, and F1 scores to better explain our model's performance. We have presented this information in a table format for clarity.

Dataset name	Task name	Precsion	Recall	F1
WebNLG	NER	0.9688	0.9808	0.9748
	RE	0.9274	0.9300	0.9287
ADE	NER	0.8908	0.9029	0.8968
	RE	0.8211	0.8087	0.8189
SCIERC	NER	0.6821	0.6690	0.6758
	RE	0.4571	0.3296	0.3830

2. We have redrawn Figure 1 based on our model's structure, and used Visio to ensure the clarity of the image, and copied the original image to the Word document.

3. It was our oversight not to mention the hardware platform used in our experiments. We have updated our paper in section 4.2.2 to include a note about the hardware platform used. Specifically, Casrel (A Novel Cascade Binary Tagging Framework for Relational Triple Extraction) and TPlinker (TPLinker: Single-stage Joint Extraction of Entities and Relations Through Token Pair Linking) were tested using the original configuration from their papers, while PFN (PFN:A Partition Filter Network for Joint Entity and Relation Extraction) and our model were tested on an A5000 hardware environment. To minimize the impact of the hardware platform, we reported the processing time for each model on a per-sample basis.

4. Thank you for pointing out this problem in the manuscript. We have revised all the images in the paper and increased their resolution to ensure that the numbers and details are more visible.

5. In order to make it easier for readers to understand, we have added an example in the fused module section of the paper to describe the working process of our model. (To illustrate how our model works, we take the sentence "The current CEO of Apple is Cook" as an example….)

6. Thank you for pointing out the spelling and grammar errors in the article. We have carefully proofread the article and corrected these errors.

We are truly grateful for your valuable suggestion, and we will keep this in mind in our future research.

 

Author Response File: Author Response.docx

Reviewer 3 Report

Review is attached as pdf file.

Comments for author File: Comments.pdf

Author Response

Thank you so much for your kind words and valuable feedback. We are grateful for your recognition of our work and have taken your suggestions seriously.

We have investigated the reasons behind the errors generated by our model, starting from the dataset itself. Insufficient manual annotation of the dataset is one of the factors, which has also been mentioned in the TPLinker (TPLinker: Single-stage Joint Extraction of Entities and Relations Through Token Pair Linking) paper. However, due to time constraints, we were not able to conduct a comprehensive evaluation of the quality of the entire dataset. We have performed experimental validation and evaluation of our paper's performance in terms of entities. We have found that our model's performance is poor for short entities (entities with only one word), and we have provided specific entity-level results. The experiment results is shown in the table. We also observe that EPRE and PFN (PFN:A Partition Filter Network for Joint Entity and Relation Extraction) on Out_triple F1 values (46.5%, 48.6%) were significantly lower than in In_triple (75.4%, 76.2%). We believe that this phenomenon is due to the low cohesion between short entities and other words, as well as the limited contextual information available for out_triple entities.

	Our		PFN
Data_length	NER	RE	NER	RE
long	p=0.6842, r=0.6802, f=0.6822	p=0.4181, r=0.2797, f=0.3352	p=0.6746, r=0.6836, f=0.6790	p=0.3727, r=0.3166, f=0.3424
Medium	p=0.6798, r=0.6434, f=0.6611	p=0.4531, r=0.2802, f=0.3463	p=0.6995, r=0.6783, f=0.6888	p=0.3720, r=0.2947, f=0.3288
Short	p=0.5600, r=0.5385, f=0.5490	p=0.2500, r=0.1868, f=0.2138	p=0.5686, r=0.5577, f=0.5631	p=0.2500, r=0.2222, f=0.2353

In addition, in order to make it easier for readers to understand, we have added an example in the fused module section of the paper to describe the working process of our model.

The introduction section has been revised to make it more appealing to readers, and we have implemented your excellent suggestion of using a table format to present related research in the paper. This has certainly enhanced the overall presentation of the previous studies in the related work section.

We have carefully revised and edited the abstract and introduction section to enhance the clarity and coherence of the paper. Additionally, we present related research in a table format, which enhances the presentation of the previous studies in the related work section. We hope that the updated version meets your expectations and look forward to hearing your feedback.

Thanks again for your time and valuable input.

Author Response File: Author Response.docx

Round 2

Reviewer 1 Report

The authors answered my questions and provided a discussion concerning my complaints. Thank you.