A Systematic Literature Review of Eye-Tracking and Machine Learning Methods for Improving Productivity and Reading Abilities

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

Thank you for the opportunity to review this manuscript titled “A Systematic Literature Review of Eye Tracking and Machine Learning Methods for Improving Productivity and Reading Abilities.”

The manuscript conducts a systematic literature review using PRISMA guidelines, analyzing 1,782 articles and narrowing down to 42 studies. It categorizes findings into four main areas: eye metric classification (using machine learning to track and classify eye movements (e.g., fixations, saccades)), measuring comprehension (how eye-tracking data correlates with reading comprehension), measuring attention (the role of eye tracking in assessing cognitive workload and focus), and typography & typesetting (how text formatting influences reading efficiency). The systematic review highlights (i) the use of CNN-LSTM models for accurate eye-movement classification, (ii) synthetic data generation (SP-EyeGAN) as a solution for data scarcity, (iii) typography modifications (e.g., bolding, spacing) for improving readability, and (iv) challenges in generalizability, bias, and data scarcity.

Although this is an interesting manuscript, here are a few suggestions for revision.

Minor Revision

The authors should address the following:

1. Research questions are not stated. The relevant question formulation framework will clarify the study’s objectives for the reader.

- I strongly recommend that the authors revise accordingly.

2. Improve Clarity in Data Presentation:

- The PRISMA flowchart should be made more visually apparent. Authors can download https://www.prisma-statement.org/s/PRISMA_2020_flow_diagram_new_SRs_v1-lml8.docx and clearly state the different exclusion criteria. Also, they can consider color-coding the different exclusion criteria.

- Furter, including tables that summarize study methodologies and key findings would enhance readability.

3. Clarify Study Selection Biases

- The paper should provide an explanation for the exclusion of specific articles beyond the PRISMA criteria. For example, were studies on dyslexia, ADHD, or cognitive impairments excluded unintentionally?

I believe that these revisions would significantly enhance the manuscript's clarity and overall quality.

Author Response

Thank you for your valuable feedback, please refer to the attached file.

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

This paper reviews the recent application of eye tracking technology and machine learning methods to improve productivity and reading ability. Through the database search of Scopus and Web of Sciences, relevant English literature from 2015 to 2024 is screened, focusing on eye movement indicators, the application of machine learning methods, and the relationship between reading efficiency and comprehension. The results show that the eye movement parameters can effectively reflect the cognitive load and understanding level during reading, and the typography characteristics also have a significant impact on reading experience. In addition, the paper explores the potential of using synthetic eye tracking data to train machine learning models. After careful review, my comments are as follows:

1. The citation order of references in the main text is a little confusing, which affects the reading. Please number the references in the order of their first citation.
2. The layout of Figure 1~ Figure 3 needs to be adjusted, and there are a lot of gaps.
3. The topic of the article is "eye tracking and machine learning methods for improving productivity and reading abilities", However, why the author took "Typography and Typsetting" into consideration when screening the articles seems to be inconsistent with the topic.
4. Whether the authors considered the level of the paper when selecting the articles.
5. The introduction part is too short to fully explain the research background, purpose and importance, so it is suggested to add relevant content to make it easier for readers to understand the significance of the research.
6. Can you provide Supplementary Table A1?
7. The use of the MMAT tool is not fully covered, so it is recommended that it be covered in detail.

Author Response

Thank you for your valuable feedback, please find our replies in the attached document.

Author Response File: Author Response.pdf

Reviewer 3 Report

Comments and Suggestions for Authors

The manuscript presents the results of a literature review of eye-tracking and ML-based methods used to improve productivity and reading skills. The topic is interesting and important, but the material presented has significant shortcomings.

1. The introduction is too general and does not adequately present the context of the study, its importance, and the rationale for addressing the topic. In addition, there is no clear statement of the objectives of the study and the theoretical and practical contributions of the authors.

2. The method of identifying the analyzed publications presented in Section 2.1 raises serious concerns. The query cited in lines 58-74 results in only 32 articles from the Scopus database, many of which were not analyzed in the paper. The manuscript lacks the analogous query the authors used for the WoS database. Therefore, the number of articles cited in line 78 (927) is questionable. To assess the reproducibility of the research, it is crucial to accurately report the queries used and the number of responses received for all databases analyzed.

3. The remainder of Section 2 inadequately presents the approach taken by the authors. The criteria given are vague, making it impossible to reliably evaluate the results presented in Section 3. The description of the method should be precise, and the decisions made to exclude articles should be well justified. A graphical presentation of the flow of the process of selecting the analyzed papers could be considered.

4. The presented results of article selection are also questionable. I wrote about the unclear origin of the original number of papers (1782) above. Further ambiguities concern the 855 articles that “were excluded due to irrelevance or duplication during the initial screening” (lines 146-147). On what basis was this decision made? How did the authors know that so many papers were irrelevant when the basis was not the title, abstract, or text (considered in subsequent stages of the process)? What duplications are we talking about when there were only 32 papers in the Scopus database searched according to the given query? In the next step—analysis of the title and rejection of 577 more articles—the authors rejected 2/3 of the papers solely on the basis of the title, without analyzing anything else (not even keywords).

In the end, the authors decided to include only 42 papers in the study, which is a small number for a systematic literature review. Adding to this the unclear criteria for the selection of articles, the reliability of the presented results seems questionable.

5. Regardless of doubts about the selection of articles for the study, the presentation of the results itself also raises concerns. In particular, there is a lack of a synthetic summary of the subject areas identified by the authors, with an indication of the papers in which they were addressed. This would have made it possible to assess trends and the incidence of various concepts.

In conclusion - although the topics taken up are important and interesting, the submitted manuscript raises too many formal and methodological doubts. 

Author Response

Thank you for your valuable feedback, please find our replies in the attached document.

Author Response File: Author Response.pdf

Round 2

Reviewer 3 Report

Comments and Suggestions for Authors

Thank you for addressing my comments and feedback. In my opinion, in its current version, the manuscript can be published.