Creating a Parallel Corpus for the Kazakh Sign Language and Learning

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

The article does not provide sufficient details about the specialized software used for annotation. Understanding the capabilities and limitations of the software is crucial, as it can significantly affect the accuracy and reliability of the annotations. A more detailed description of the software, including its features and any potential biases or errors, would enhance the credibility of the study.

Some captions of the tables and figures are unclear or do not make sense. For example, Table 1 is labeled "Reversible," Table 2 is labeled "Local," and Table 3 is labeled "Animate." Additionally, Figure 1, titled "The grammatical structure of the Kazakh sign language," contains unreadable characters. The left part of Figure 2 (should be fig3 in order), which is supposed to illustrate the "Parallel corpus," is blurred. These issues significantly impact the readability and comprehension of the article.

There is no mention of user testing or feedback from the target community, specifically individuals with hearing and speech impairments. User testing is essential to ensure that the developed resources are practical, user-friendly, and beneficial in real-world applications. Including feedback from the community would provide valuable insights and help refine the corpus to better meet their needs.

The article fails to address how consent was obtained from the participants or how their privacy was protected. Ethical considerations are critical, especially when dealing with vulnerable communities. Ensuring that all participants provided informed consent and that their privacy was safeguarded is not only a legal requirement but also a moral obligation. A section detailing the ethical framework and consent process would strengthen the study's integrity.

The article claims that the corpus will support the teaching of KSL, but it does not discuss how this integration will be achieved. Details about the educational framework and the steps to incorporate the corpus into existing teaching programs are necessary. This includes information on how educators will be trained to use the corpus, how it will be integrated into the curriculum, and how its effectiveness will be measured.

Comments on the Quality of English Language

The English language quality in several sections is poor, making it difficult to understand the content. 

Author Response

Hello, dear reviewer! Thank you in advance for your work. We have worked out all the points on the review and are sending the response as a file.

Author Response File: Author Response.docx

Reviewer 2 Report

Comments and Suggestions for Authors

The article discusses the connection of spoken language with Kazakh sign language and is worth to be processed. Before publication, I have some remarks:

1.      There is a description of the grammatical structure of this language (Fig. 1). This diagram is not very legible for the reader. The description of this figure should be expanded.

2.      In the first table SVO, SOV appear. These concepts should be written in brackets in the table description.

3.      Figure 2 also requires a more extensive description.

4.      The second figure, which should be Figure 3, is completely illegible. It is not known what is on the left side of this figure, as well as in the second column, in the first row, it is difficult to read any data.

5.      In part 9 the authors refer to the Bleu criterion (written once in small letters and once in capital letters). This criterion should be described in the "Materials and Methods" section.

6.      Also, I'd like to point out if the proposed approach can be implemented in other languages/groups of languages. How other nations can benefit from the proposed approach?

7.      The literature contains insufficient new (from the last two years) articles. The Introduction should be expanded by referring to other works such as:

·         https://doi.org/10.55041/ijsrem34023.

·         https://doi.org/10.1109/IITCEE59897.2024.10467410.

·         https://doi.org/10.1007/s10209-023-00992-1.

Author Response

Hello, dear reviewer! Thank you in advance for your work. We have worked out all the points on the review and are sending the response as a file.

Author Response File: Author Response.docx

Reviewer 3 Report

Comments and Suggestions for Authors

The article is dedicated to the creation of a parallel corpus of Kazakh Sign Language (KSL) using machine learning methods and deep neural networks for automatic translation between Kazakh written and sign language. The topic of the article is relevant. The structure follows the format accepted in MDPI for research articles (Introduction, including an analysis of related works; Models and Methods; Results; Discussion; Conclusions). The level of English is acceptable, and the article is easy to read. The figures in the article are of acceptable quality. The article cites 20 sources, many of which are outdated. The References section is formatted carelessly.

The following comments and recommendations can be made regarding the article:

1. The study presents insufficient new information since the developed parallel corpus of Kazakh Sign Language (KSL) is built upon already known glossing and machine learning methods. The article lacks a clear comparison with existing solutions and does not explicitly demonstrate what makes it novel compared to similar studies.

2. Analytical modeling is presented in a limited manner. The article does not include formal mathematical models of data processing, does not describe the text preprocessing process, and does not provide equations or algorithmic explanations that would help better understand the mechanism of the proposed method. The choice of the Transformer architecture is not supported by a comparative analysis with alternative methods such as LSTM or CNN, making it difficult to assess the justification of this choice.

3. Inconsistencies and contradictions are present in the text. For example, in the data section, it is stated that the study is based on 2,000 sentences, but later it is claimed that more than 100,000 sentences were used for model training. However, there is no explanation of how such a volume of data was obtained. Additionally, terminology is sometimes used incorrectly, which may cause confusion, such as the phrase "parallel building" instead of the correct "parallel corpus."

4. The experimental setup is not sufficiently rigorous. The study does not include a comparative analysis of the model's performance on different subsets of data, such as specific categories of sentences (reversible, locative, animate, etc.). Furthermore, there are no experiments demonstrating the system’s effectiveness with real users, which would confirm its practical applicability.

5. The statistical analysis of results is superficial. The translation quality is evaluated solely using the BLEU metric, which does not account for the syntactic and semantic differences between sign and written languages. Moreover, the article does not present confidence intervals or statistical tests that would help assess the significance of the results. Typical model errors are not analyzed, making it difficult to identify its weaknesses and potential areas for improvement.

6. The practical value of the study is not sufficiently demonstrated. The authors do not provide examples of text translation into Kazakh Sign Language, preventing an assessment of the quality of the developed parallel corpus. Additionally, the article does not discuss the potential integration of the model into real-world applications such as educational programs or automatic translation systems, making its practical significance unclear.

7. The structure of the article has shortcomings. There are duplicated sections in the text, for example, "Discussion" and "Conclusions" partially repeat each other. The abstract is formulated too generally and does not disclose the research methodology, making it difficult to understand the study's key findings. Visual elements such as tables and graphs are not always accompanied by sufficient explanations, reducing their informativeness.

Author Response

Hello, dear reviewer! Thank you in advance for your work. We have worked out all the points on the review and are sending the response as a file.

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

The scientific contribution and writing quality of the whole manuscript have been improved according to the peer review suggestions. Suitable for publication after format editing.

Author Response

Dear reviewer! We thank you for your work and appreciation!

Reviewer 2 Report

Comments and Suggestions for Authors

The paper is sufficiently improved and in y opinion, it is ready to be published

Author Response

Dear reviewer! We thank you for your work and appreciation!

Reviewer 3 Report

Comments and Suggestions for Authors

Dear Authors,

Below, I provide an analysis of the changes you have made and an evaluation of whether the revisions sufficiently address the concerns raised.

1. In my initial review, I raised concerns about the study's novelty, particularly in comparison to existing approaches to sign language corpus development and machine translation. Your response acknowledges these concerns and emphasizes that the Kazakh Sign Language (KSL) parallel corpus represents a significant achievement for linguistic research in Kazakhstan. You have also clarified the adaptations made to existing machine learning methods and introduced a discussion on the limitations of prior work. However, while you have included this discussion, the paper still lacks a clearly defined comparative analysis with existing solutions. The response claims that such a comparison has been made (lines 623-635), but the manuscript does not explicitly outline how the proposed approach outperforms or differs fundamentally from previous methodologies.
2. Initially, I pointed out the insufficient formalization of the proposed approach, including a lack of mathematical models and algorithmic explanations. Your revised manuscript now provides a more structured description of the data processing pipeline (lines 361-385) and introduces word order transformation rules. Additionally, the formal definition of syntactic dependencies, animacy classification, and tense adjustments strengthens the methodological foundation. However, the justification for the choice of Transformer architecture remains underdeveloped. While you provide some theoretical advantages of Transformers over alternatives like LSTMs or CNNs, a direct empirical comparison is still missing.
3. A key issue in the original manuscript was the contradictory data reporting, particularly regarding the number of analyzed sentences. You have now clarified the distinction between the 500 analyzed sentences (used for linguistic analysis) and the 100,000 training sentences (used for model training). The revised text (lines 285-287, 456-459, 492-498) resolves this confusion, making the data sources and experimental setup more transparent. Additionally, terminology inconsistencies (such as "parallel building" instead of "parallel corpus") have been corrected.
4. The original review noted that the experimental section lacked robustness, as there was no comparative analysis on different data subsets (e.g., reversible vs. non-reversible sentences) or real-world testing with end users. Your revisions now include a more detailed breakdown of sentence structure (Tables 1-4) and highlight variations in word order based on linguistic features (lines 277-360). However, no real-user evaluation of the model’s effectiveness is provided. While computational metrics such as BLEU scores have been used, the lack of human evaluation remains a significant limitation.
5. Previously, I pointed out that BLEU scores alone are insufficient to fully assess translation quality, as they do not account for syntactic or semantic differences between sign and written languages. Your revised manuscript provides additional details about BLEU calculations and presents a comparative BLEU score analysis between Seq2Seq and Transformer models (Figure 6). This is a valuable addition. However, confidence intervals, statistical significance tests, or an error analysis of typical translation mistakes are still missing.
6. One major concern was the lack of evidence demonstrating the practical usability of the proposed system. Your revision now references the development of a mobile application (Word2Sign), which translates text into gloss and uses an avatar to display sign language (link provided to the App Store). This is a strong practical contribution that enhances the impact of your work. However, quantitative user engagement data, usability testing, and comparisons with existing applications are not provided, which weakens the practical evaluation.
7. I initially pointed out structural weaknesses, including redundant sections and a vague abstract. The revised manuscript successfully eliminates redundancies between the "Discussion" and "Conclusion" sections. Additionally, the abstract now better reflects the methodology and key findings. Graphical explanations of tables and figures have also been improved.

Author Response

Dear Reviewer, we thank you for your work. We inform you that all recommendations and suggestions have been taken into account, and all comments have been reviewed!

Author Response File: Author Response.docx

Round 3

Reviewer 3 Report

Comments and Suggestions for Authors

Dear Authors,

Below, I provide an analysis of the revisions of the manuscript.

1. The initial concern regarding the lack of explicit novelty compared to existing methods was partially addressed. The revised manuscript now highlights how the parallel corpus was adapted for Kazakh Sign Language (KSL) by incorporating specialized rules and a custom parser. However, while the paper discusses the importance of this adaptation, a direct comparison with prior studies still lacks depth. A clearer demonstration of how this work advances beyond existing methods remains necessary.
2. One of the key criticisms was the limited formalization of the analytical modeling process. The authors have responded by including more equations, an algorithmic description, and details on preprocessing and sentence transformation. This is a significant improvement, making the methodology clearer. However, the justification for choosing the Transformer model over alternatives (e.g., LSTM, CNN) remains underdeveloped. Although BLEU scores show improved results, a more comprehensive comparative analysis (e.g., statistical validation of performance differences) would strengthen the argument.
3. The original version contained inconsistencies, particularly regarding the number of sentences used in analysis and model training. The revised manuscript provides clarification, distinguishing between the 2,000 sentences used for grammatical analysis and the over 100,000 sentences used for training the Transformer model. The terminology has been corrected, with "parallel corpus" consistently used instead of incorrect expressions such as "parallel building." These revisions resolve the issues effectively.
4. The study now includes a more structured analysis of sentence categories (reversible, locative, animate, inanimate, etc.), which improves transparency. However, the experimental setup remains somewhat incomplete. There are still no real-user experiments to validate the effectiveness of the system in practical settings. Moreover, while BLEU scores are used as an evaluation metric, the limitations of BLEU in sign language translation are not discussed in depth, and alternative metrics or human evaluation are not included.
5. The manuscript now describes a mobile application developed for text-to-KSL translation, which strengthens the practical contribution. However, further details on user engagement, adoption, and performance in real-world scenarios would be beneficial.
6. Structural issues, such as redundancy between the "Discussion" and "Conclusion" sections, have been largely resolved. The abstract has been improved to include more details about the methodology, making it more informative. Tables and figures are now better explained, enhancing clarity.

Author Response

Dear reviewer, we sincerely thank you for your feedback on our work and valuable recommendations. We have analyzed all your comments and suggestions, made the necessary edits and supplemented the manuscript in accordance with your recommendations.

Author Response File: Author Response.docx

Round 4

Reviewer 3 Report

Comments and Suggestions for Authors

I would like to thank the authors for their efforts in revising the article and for their detailed responses to the comments. The presented corrections have significantly enhanced the quality of the research and its presentation.

In the new version of the article, the section on analytical modeling has been considerably expanded, and formal descriptions of the data processing process have been added, including syntactic dependencies, word permutation, and temporal markers. This makes the research methodology more transparent and reproducible.

Previously identified discrepancies regarding the volume of data used have been resolved: the origin of the corpus containing over 100,000 sentences is now clearly specified. This strengthens the foundation of the experimental framework and makes the presented results more reliable.

The experimental section has also been substantially improved. A detailed analysis of the sentence structure in Kazakh Sign Language (KSL) has been included, supplemented with a statistical description of word order depending on animacy, reversibility, and other factors. The presented results demonstrate the depth of linguistic analysis and comply with current standards in the field of natural language processing.

Furthermore, the practical significance of the work is now more clearly justified. Information on the development of a mobile application demonstrating the real-time application of the proposed method has been included. This confirms the relevance of the research and its potential benefit to the KSL user community.

The structure of the article has also been improved: redundant sections have been removed, the abstract has been clarified, and descriptions for tables and figures have become more informative, contributing to a better understanding of the material.

Although the inclusion of additional translation quality metrics and comparisons with alternative methods could strengthen the article, the presented results are sufficient for acceptance. Given the significant improvements and the resolution of critical comments, I recommend the article for publication.