Research on a Joint Extraction Method of Track Circuit Entities and Relations Integrating Global Pointer and Tensor Learning

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

The paper is well organized. And the results have prove its novelty and efficiency.  I suggest the authors may compare their performance with some Chinese public datasets that are more similar to their actual situation.

Author Response

Comments 1: I suggest the authors may compare their performance with some Chinese public datasets that are more similar to their actual situation.

Response 1: Thank you very much for your valuable suggestion. We agree with this comment. Therefore, we have added a comparative experiment using the Chinese public dataset DuIE, which is more aligned with our actual application scenario. We have also introduced the English public datasets NYT and WebNLG, as well as the Chinese public dataset DuIE from Baidu. The results of this comparison can be found in Section 4.4, page 13, Table 4, with a detailed discussion to highlight the model's performance in a context closer to our intended application.

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

The topic is certainly interesting and current. The availability of digitally structured diagnostic data is important for the predictive maintenance of track circuits that are the fundamental entities for the safety and control of railway traffic.

The possibility of having diagnostic data collected in free text format in a structured and digitalized way could make an important contribution to increasing the reliability of track circuits and avoiding failures that produce negative effects on circulation.

The idea proposed in the study to automate the interpretation of unstructured diagnostic data is important.

The study focuses mainly on the analysis of the effectiveness of the extraction models.

The approach is original especially for the choice of the models examined and the proposal for their integration.

The evaluation of the sustainability and correctness of the proposed models requires specific expertise of specialists in the sector rather than railway engineers.

However, the article is useful for railway engineers to evaluate the applicability of the proposed model to the command and control systems of railway traffic.

The article is well structured.

To facilitate the evaluation by readers who are not particularly expert in extraction models, it would be useful to integrate the discussion with brief reminders of theoretical models and examples related to the field of application of binary circuits.

The results seem to confirm the validity of the proposed model.

I report below some suggestions • Figure 2: recall the meaning of the symbols, add examples as reported in figure 1 • Figure 3 clarify the symbols • Clarify NYT and WebNLG, F1 • Tab 3 clarify Prev. Ric. • Figure 6 clarify the meaning of Epoch 

Author Response

Comments 1: Figure 2: recall the meaning of the symbols, add examples as reported in figure 1.

Response 1: Thank you for your valuable suggestion. We fully agree with your assessment. To clarify the meaning of each symbol in Figure 2, we have provided a detailed description of each symbol in Table 1, located in Section 3 on page 5. Presenting this information in a tabular format enhances clarity and organization.

Comments 2: Figure 3 clarify the symbols.

Response 2: Thank you for pointing out this issue. We have provided a detailed description of each symbol in Table 1, located in Section 3 on page 5, to facilitate a clear and intuitive understanding of each symbol.

Comments 3: Clarify NYT and WebNLG, F1.

Response 3: Thank you for your valuable feedback. Both NYT and WebNLG are English public datasets. The NYT dataset was created by aligning New York Times text with the Freebase knowledge base, while the WebNLG dataset was originally developed for the Natural Language Generation (NLG) task. We have also included comparative experiments with the DuIE Chinese public dataset from Baidu. A detailed description of these datasets can be found in Section 4.4 on page 13, with their sources cited in the references. The F1 score, which represents the harmonic mean of precision and recall, is explained in Section 4.2, and its definition is provided in Equation (27).

Comments 4: Tab 3 clarify Prev. Ric.

Response 4: Thank you for your valuable feedback. We agree that using "Prec." for precision and "Rec." for recall may cause ambiguity. We have revised these abbreviations and provided a detailed explanation of Precision (P), Recall (R), and F1 score in Section 4.2, along with the corresponding formulas for clarity.

Comments 5: Figure 6 clarify the meaning of Epoch.

Response 5: Thank you for your suggestion. Here is a more detailed explanation: one epoch represents a complete pass through the entire dataset during training. In the comparative experiments on the track circuit dataset, we conducted 50 training epochs to fully optimize the model’s performance. We have added a clarification of "epoch" in Section 4.5.

Author Response File: Author Response.pdf

Reviewer 3 Report

Comments and Suggestions for Authors

The paper introduces a joint extraction model for entities and relations within track circuit maintenance data using advanced neural network techniques. The authors employ a multi-level semantic fusion encoder that leverages RoBERTa for contextual understanding, a tensor learning module utilizing Tucker decomposition for efficient relation representation, and an Efficient Global Pointer module that refines entity identification. The paper is well organized. However, I will suggest some points that need to be addressed to improve the paper.

 

1.      The introduction should be modified so that the last paragraphs should be divided into paper contributions and finds and paper organization as separated subsections.

2.      The contribution is not clear in comparison to the state-of-the-art methods in the introduction. I would suggest making it clear what has been done in comparison to the state-of-the-art methods showing the contribution of the paper.

3.      There is a non-English statement in Table 2. Track Circuit Entity-Relation Matching Template (Partial). You must remove it. Please modify.

4.      Some grammar errors have been identified please correct them.  

5.      Noting that comma or dots should be stated after the equations.

6.      The author needs to state the paper's findings in the last part of the introduction. For example, what has been achieved in the results? As stated in the abstract.

7.      Despite implementing class-balancing techniques, the model may still face challenges with data imbalance, as the positive and negative samples in real-world maintenance data could vary significantly. Further enhancement of balancing techniques or implementation of dynamic sampling methods might improve generalizability and robustness across datasets with different distributions.

8.      Testing the model on actual railway maintenance logs and fault records could provide insights into its practical performance, allowing for adjustments to better handle domain-specific data.

9.      The model shows sensitivity to parameter tuning, particularly in the dilation rates for convolutional layers and hyperparameters within the tensor learning module. Implementing automated hyperparameter tuning methods, such as Bayesian optimization, could reduce the reliance on manual tuning and optimize the model’s performance consistently.

10.  The model might struggle with highly contextual or ambiguous language within maintenance records, affecting its extraction accuracy. Integrate contextual embedding layers or fine-tune the model with additional semantic information specific to railway maintenance terminology to improve its ability to handle domain-specific expressions.

11.  Figure 4 contains non-English words please modify.

 

 

 

 

 

Comments on the Quality of English Language

Need to be improved 

Author Response

Comments 1: The introduction should be modified so that the last paragraphs should be divided into paper contributions and finds and paper organization as separated subsections.

Response 1: Thank you for your valuable suggestion. We have revised the introduction section and added the following “Paper Organization” part:

“The rest of the paper is structured as follows: Section 2 introduces the foundational techniques of early knowledge extraction and reviews the recent research progress in knowledge extraction within the railway domain. Section 3 provides a detailed explanation of the joint extraction model proposed in this paper. Section 4 presents the experimental setup and result analysis, including the visualization of knowledge extraction results from real-world track circuit case data. Section 5 summarizes the main contributions of this study.”

This adjustment provides a clearer presentation of the study's contributions and enhances the paper's readability. The revised sections can be found in the introduction.

Comments 2: The contribution is not clear in comparison to the state-of-the-art methods in the introduction. I would suggest making it clear what has been done in comparison to the state-of-the-art methods showing the contribution of the paper.

Response 2: Thank you for your insightful comment. We agree that clarifying our contributions relative to state-of-the-art methods is essential. In response, we have updated the introduction to explicitly compare our approach with recent advanced methods, highlighting our unique contributions. Specifically, we emphasize the model’s ability to handle overlapping entity relationships, leverage multi-level semantic fusion, and utilize tensor learning for improved performance. These points clarify how our method advances beyond existing approaches. The updated comparison can be found in the introduction section.

Comments 3: There is a non-English statement in Table 2. Track Circuit Entity-Relation Matching Template (Partial). You must remove it. Please modify.

Response 3: Thank you for pointing this out. We have removed the non-English text in Table 2, "Track Circuit Entity-Relation Matching Template (Partial)," and replaced it with the corresponding English terms to ensure consistency throughout the paper. The updated Table 2 has been renumbered as Table 3 and can be found in Section 4.3 on page 12.

Comments 4: Some grammar errors have been identified please correct them.  

Response 4: Thank you for highlighting this. We have thoroughly reviewed the manuscript and corrected the identified grammatical errors to improve clarity and readability. The changes are marked in red throughout the revised document.

Comments 5: Noting that comma or dots should be stated after the equations.

Response 5: Thank you for your attention to detail. We have reviewed the manuscript and ensured that commas or periods are placed appropriately after each equation, following standard formatting guidelines. These changes are reflected in the revised document.

Comments 6: The author needs to state the paper's findings in the last part of the introduction. For example, what has been achieved in the results? As stated in the abstract.

Response 6: Thank you for your valuable suggestion. To improve the flow of the introduction and clearly present the paper’s findings, we have added a statement on the study's results at the beginning of the contributions section. This section now outlines the purpose of the study, followed by the achieved F1 scores on public datasets NYT, WebNLG, and DuIE (92.1%, 92.7%, and 78.2%, respectively), as well as validation on a private track circuit dataset, demonstrating the model's potential in real-world maintenance scenarios. We believe this adjustment provides a clearer presentation of the study's findings within the introduction.

“To provide intelligent decision-making support for on-site maintenance personnel, accelerate the fault detection and repair process, and achieve automated knowledge graph construction for a shared knowledge base, this study proposes a joint extraction model for track circuit entities and relations, integrating global pointer and tensor learning. The model achieved F1 scores of 92.1%, 92.7%, and 78.2% on the public datasets NYT, WebNLG, and DuIE, respectively, and was further validated on a private track circuit dataset, demonstrating its potential in real-world maintenance scenarios. The innovations and contributions of this study include:

(1) Unlike existing models that only use top-layer outputs from pre-trained models, this research uses multi-layer dilate gated convolutional neural network (MDGCNN) to extract features from the 12-layer RoBERTa-wwm encoder output. These 12 different levels of semantic information are then adaptively weighted and fused, enhancing the model’s feature representation and improving the recognition accuracy of complex entities and relations.

(2) Existing methods often neglect the deep correlation between multiple relations, especially when multiple entities overlap. To address this, this study applies Tucker decomposition to learn and reconstruct core tensors, subject and object factor matrices, and relation weight matrices, resulting in high-dimensional relation tensors. This not only improves the accuracy of extracting overlapping relations but also strengthens the semantic correlation modeling between different relation types.

(3) The model adopts the efficient global pointer and introduces rotary position encoding (ROPE) along with a multiplicative attention mechanism to accurately compute entity start and end positions. By sharing weights, the model reduces the number of parameters while maintaining high recognition performance and reducing time complexity.

The rest of the paper is structured as follows: Section 2 introduces the foundational techniques of early knowledge extraction and reviews the recent research progress in knowledge extraction within the railway domain. Section 3 provides a detailed explanation of the joint extraction model proposed in this paper. Section 4 presents the experimental setup and result analysis, including the visualization of knowledge extraction results from real-world track circuit case data. Section 5 summarizes the main contributions of this study.”

Comments 7: Despite implementing class-balancing techniques, the model may still face challenges with data imbalance, as the positive and negative samples in real-world maintenance data could vary significantly. Further enhancement of balancing techniques or implementation of dynamic sampling methods might improve generalizability and robustness across datasets with different distributions.

Response 7: Thank you for your thoughtful suggestion. We agree that data imbalance in real-world maintenance data can present challenges due to significant variation between positive and negative samples. To address this, we have implemented class-balancing techniques as referenced in [34] and [36], which have been shown to effectively enhance generalizability and robustness. Additionally, we conducted comparative experiments on three public datasets, including a Chinese dataset, and observed strong performance across all, demonstrating the model's ability to generalize well across datasets with varying distributions. Given these results, we believe our current approach to class balancing is sufficient for maintaining the model’s stability and effectiveness without further enhancements at this stage.

Supplement:

[34] Wang, Z.; Nie, H.; Zheng, W.; Wang, Y.; Li, X. A novel tensor learning model for joint relational triplet extraction. IEEE Trans. Cybern. 2023, 54, 2483-2494. https://doi.org/10.1109/TCYB.2023.3265851

[36] Su, J.; Murtadha, A.; Pan, S.; Hou, J.; Sun, J.; Huang, W.; Wen, B.; Liu, Y. Global Pointer: Novel Efficient Span-based Approach for Named Entity Recognition. arXiv 2022, arXiv:2208.03054.

Comments 8: Testing the model on actual railway maintenance logs and fault records could provide insights into its practical performance, allowing for adjustments to better handle domain-specific data.

Response 8: Thank you for your valuable suggestion. We recognize the importance of evaluating the model's performance on actual railway maintenance logs and fault records to gain practical insights. In response, the track circuit experiments in Section 4.5 are based on actual maintenance data from the past five years across regions such as the Beijing and Shanghai Railway Bureaus. This evaluation closely aligns with domain-specific application scenarios, effectively demonstrating the model's robustness and generalizability. We believe this approach offers meaningful insights into the model’s application in real-world settings without requiring further adjustments at this stage.

Comments 9: The model shows sensitivity to parameter tuning, particularly in the dilation rates for convolutional layers and hyperparameters within the tensor learning module. Implementing automated hyperparameter tuning methods, such as Bayesian optimization, could reduce the reliance on manual tuning and optimize the model’s performance consistently.

Response 9: Thank you for this insightful suggestion. We agree that automated hyperparameter tuning methods could further refine model performance. In our study, however, the tuning of the tensor learning module’s hyperparameters was based on well-established guidelines in the literature, specifically referenced from [34], while the dilation rate tuning for the convolutional layers followed the approaches outlined in [28] and [47]. These references provided effective tuning strategies that allowed us to achieve stable and robust results. Given the current model's satisfactory performance and stability, we believe that our manual tuning approach is sufficient at this stage.

Supplement:

[28] Kan, Z.; Qiao, L.; Yang, S.; Liu, F.; Huang, F. Event arguments extraction via dilate gated convolutional neural network with enhanced local features. IEEE Access 2020, 8, 123483-123491. https://doi.org/10.1109/ACCESS.2020.3004378

[34] Wang, Z.; Nie, H.; Zheng, W.; Wang, Y.; Li, X. A novel tensor learning model for joint relational triplet extraction. IEEE Trans. Cybern. 2023, 54, 2483-2494. https://doi.org/10.1109/TCYB.2023.3265851

[47] Yu, F.; Koltun, V. Multi-scale context aggregation by dilated convolutions. arXiv 2015, arXiv:1511.07122.

Comments 10: The model might struggle with highly contextual or ambiguous language within maintenance records, affecting its extraction accuracy. Integrate contextual embedding layers or fine-tune the model with additional semantic information specific to railway maintenance terminology to improve its ability to handle domain-specific expressions.

Response 10: Thank you for this valuable suggestion. We recognize that highly contextual or ambiguous language in maintenance records may impact extraction accuracy. In our study, however, we address this challenge through a multi-layer semantic fusion approach in the upstream encoding model. By adaptively weighting and combining the 12 layers of RoBERTa, each progressively capturing deeper semantic information, our model achieves multi-granularity semantic integration. This approach enables the model to better handle domain-specific expressions and contextual complexity without requiring additional contextual embedding layers or further fine-tuning. We believe this current setup effectively meets the needs of domain-specific language handling.

Comments 11: Figure 4 contains non-English words please modify.

Response 11: Thank you for pointing this out. We have revised Figure 4, replacing all non-English words with their English equivalents to ensure consistency and clarity throughout the paper. The updated figure can be found in Section 3.2, Figure 4 of the revised manuscript.

Author Response File: Author Response.pdf