KWM-B: Key-Information Weighting Methods at Multiple Scale for Automated Essay Scoring with BERT

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

Well-structured and presented work. PLease correct the following four minor points:

1) Line 93 is "orphan"; move to the next page, if possible.

2) Equation (10) is not properly connected to the text (and maybe (11) as well). Probably you have to change the "." of line 250 to a ":". Check the inclusion of the equations to the narration of your texts.

3) Unify lines 258 and 259, as well as 404 and 405.

4) Change in line 449 the "." after "dataset" probably to a ","

 

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Reviewer 2 Report

Comments and Suggestions for Authors

In this article, the authors offer many key information weighting approaches to enhance article representation at various scales, and then introduce our framework KWM-B to tackle the AES problem. Classifiers can score articles more accurately by employing these several ways to increase the weight of relevant information in their representation at different scales. The experimental results reveal that authors framework outperforms current mainstream models, indicating that our study significantly improved the article embedding representation of existing models, particularly in the scoring of argumentative writings. In addition, authors data augmentation strategy improves the model's performance on AES tasks. Authors mix data, translate and reverse it, and interchange material to help the model comprehend the differences between good and non-excellent articles. Authors findings will supplement the theoretical contributions to article embedding in automated essay grading and other challenges. This has to assist policymakers and investors in better recognizing the potential of automated essay scoring, hence accelerating the inclusion of mature automated essay scoring systems into educational and testing procedures. The structure of the article is good. Also the literature used is appropriate and have not objections to the overall feeling of the article after reading.

Author Response

Please see the attachment

Author Response File: Author Response.docx

Reviewer 3 Report

Comments and Suggestions for Authors

How sensitive are the results to the metric introduced in equation 12 (quadratic). What if L1 metric is used?

What if two essays each consisting of 3 parts (a1,b1,c1) and a2,b2,c2) are presented but in some other order, like a1,c1,b1 and a2,c2,b2 are the results different? Can the system pick it up?

 

what if you present a1,b2,c1 and a2,b1,c2 or some other combination from two different essays, what would be the results

 

please add a simple example of score calculation

 

Comments on the Quality of English Language

The paper is very dense and somewhat difficult to read. The authors need to address the issues suggested above

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Reviewer 4 Report

Comments and Suggestions for Authors

In this paper the authors present a framework based on BERT, which adopts the multi-scale key information weighting method for Automatic Paper Scoring (AES). The experimental results show that the proposed models performs better compared with other models. The following revisions are recommended for the paper:

1. In Section 2, include a table comparing the current popular methods based on architecture, advantages, limitations, training complexity, speed, and suitable applications.

2. In Section 4.2 (Implementation Details), provide hardware and software specifications along with training times to assess feasibility. Summarizing these details in a table, including memory constraints, would improve clarity and readability.

3. Thoroughly proofread the manuscript to ensure it complies with the journal's formatting guidelines before submission for publication."

 

Author Response

Please see the attachment.

Author Response File: Author Response.docx