N-Tuple Network Search in Othello Using Genetic Algorithms

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

The paper introduces dynamic n-tuple networks that adapt to different stages of the game, optimizing their evaluation accuracy as the game progresses. This is achieved through the use of Distributed Genetic Algorithms (DGA) and Biased Random-Key Genetic Algorithms (BRKGA), which efficiently optimize the shapes of n-tuple networks for specific phases of the game, such as early, mid, and late stages. The proposed approach significantly outperforms traditional static systems like Edax, achieving a 75% win rate, demonstrating its effectiveness in enhancing gameplay strategies. Additionally, the framework is designed to be game-agnostic, with the potential to be applied to other games such as Connect-Four or 2048, broadening its applicability beyond Othello.

 1. The paper claims that the proposed method is game-agnostic and could be applied to games like Connect-Four or 2048, but this generalizability is not demonstrated experimentally. Testing the approach on additional games and discussing the specific characteristics that make the method transferable would provide stronger evidence for this claim.

2. The networks are evaluated using simulated gameplay and pre-recorded datasets, but they have not been tested in live competitive environments. Conducting real-time gameplay evaluations, such as tournaments or matches against human players, would validate the system's practical effectiveness and adaptability under dynamic and unpredictable conditions. This could be mentioned as potential future work.

3.   The paper does not provide an evaluation of the computational cost or scalability of the method. Future work should analyze the resource requirements, runtime, and memory usage of the optimization process, especially for larger datasets, complex games, or varying network sizes, compared to the existing benchmarks. This would help assess the feasibility of deploying the method in real-world scenarios and its performance under more demanding conditions.

 

4. The paper claims that the proposed method is game-agnostic, with potential applicability to other games like Connect-Four and 2048, but this generalizability is not demonstrated experimentally or discussed in sufficient detail. Adapting the approach to other games would require addressing differences in game dynamics, such as gravity in Connect-Four or numeric merging in 2048, which may necessitate redesigning tuple selection and fitness functions. Additionally, the scalability of the method to larger state spaces, as in Chess or Go, and the availability of training datasets for less-documented games remain significant challenges. The authors could discuss this aspect in more detail and may suggest it as potential future work.

Author Response

Dear Reviewer,

We have followed all the suggestions provided. We have highlighted all your comments. We have also written our response below each comment.

1. The paper claims that the proposed method is game-agnostic and could be applied to games like Connect-Four or 2048, but this generalizability is not demonstrated experimentally. Testing the approach on additional games and discussing the specific characteristics that make the method transferable would provide stronger evidence for this claim.
4. The paper claims that the proposed method is game-agnostic, with potential applicability to other games like Connect-Four and 2048, but this generalizability is not demonstrated experimentally or discussed in sufficient detail. Adapting the approach to other games would require addressing differences in game dynamics, such as gravity in Connect-Four or numeric merging in 2048, which may necessitate redesigning tuple selection and fitness functions. Additionally, the scalability of the method to larger state spaces, as in Chess or Go, and the availability of training datasets for less-documented games remain significant challenges. The authors could discuss this aspect in more detail and may suggest it as potential future work.

[Response]
We have added a discussion on the redesign of the fitness function and an example of a fitness function that is not dependent on the data set.

> In such cases, it may be necessary to redesign the fitness function. While this paper defined the fitness function as the prediction accuracy on a dataset, it allows us to design the fitness function as match performance in reinforcement learning, as demonstrated in Experiment (5.3). This approach, although computationally more expensive, it eliminates the need for prior knowledge in the form of a dataset, allowing for broader application to a wider range of problems.

2. The networks are evaluated using simulated gameplay and pre-recorded datasets, but they have not been tested in live competitive environments. Conducting real-time gameplay evaluations, such as tournaments or matches against human players, would validate the system's practical effectiveness and adaptability under dynamic and unpredictable conditions. This could be mentioned as potential future work.

[Response]
We have added a description about the need for a broader range of evaluations.

> In order to verify its applicability and scalability, it needs to be evaluated in a wider range of environments, not just compared to a single agent as was done in this study.

3.   The paper does not provide an evaluation of the computational cost or scalability of the method. Future work should analyze the resource requirements, runtime, and memory usage of the optimization process, especially for larger datasets, complex games, or varying network sizes, compared to the existing benchmarks. This would help assess the feasibility of deploying the method in real-world scenarios and its performance under more demanding conditions.

[Response]
We have added a description of scalability, using the example of applying it to a larger state space.

> In addition, when applied to larger state spaces, such as Go, the network size should be larger. In order to verify its applicability and scalability, it needs to be evaluated in a wider range of environments, not just compared to a single agent as was done in this study.

Thank you very much for reviewing our work.

Best regards,

Hiroto Kuramitsu
Graduate School of Science and Technology
Tokyo University of Science

Reviewer 2 Report

Comments and Suggestions for Authors

While this paper presents an innovative application of genetic algorithms for optimizing n-tuple network shapes in the context of Othello, I find the definitions of key terms and concepts to be insufficiently clear. Specifically, the game of Othello, which serves as a foundation for much of the analysis, has not been adequately defined, making it challenging for readers unfamiliar with the game to fully understand the context and implications of the findings. Similarly, some mathematical terms and formulations are introduced without sufficient explanation or context, which may hinder accessibility to the broader audience.

Specifically,

• Definition of Terms: The manuscript lacks clear definitions for key terms, particularly those related to the n-tuple network and its application in Othello. For instance, the term "network shape" is introduced without sufficient context or clarity, leaving the reader to infer its precise meaning. Similarly, the description of "genetic algorithms" and "tuples" could benefit from more detail to make the paper accessible to a broader audience.

• Explanation of Othello: The paper assumes prior knowledge of Othello's mechanics and strategy. A brief introduction to the game, its rules, and why certain features (e.g., corners, edges) are significant would enhance reader understanding and contextualize the study.

• Mathematical Clarity: The mathematical explanations, such as Equations (1) and (2), are presented without adequate interpretation or examples. This makes it challenging for readers unfamiliar with these concepts to follow the methodology and results. Including more detailed explanations and practical examples would improve clarity.

• Structure and Presentation: Although the paper has merit, the organization could be improved. For example, Section 5 introduces experiments without fully setting up the theoretical groundwork, and there is limited discussion on the broader implications of the findings.

Given these issues, I cannot recommend the publication of this paper in its current form. I would encourage the authors to make significant revisions to define their terms clearly, provide a detailed introduction to Othello as it pertains to the study, and ensure all mathematical concepts are accessible and well-explained. 

 

 

Comments on the Quality of English Language

The English language used in the manuscript is generally clear and comprehensible. However, there are instances where sentence structures could be simplified or clarified to improve readability. For example:

1. Technical Jargon: Certain terms and phrases (e.g., "n-tuple network shape" and "fitness evaluation in genetic algorithms") are used without adequate explanation or context. Simplifying these terms or providing a glossary would make the paper more accessible.

2. Ambiguity: Sentences occasionally lack precision, leading to ambiguity. For instance, phrases like "network shape optimization is performed" could be rephrased to specify the exact process or result.

3. Consistency: Some technical terms are introduced inconsistently, making it harder for readers to follow the flow of ideas. Ensuring consistency in terminology throughout the paper would enhance clarity.

Overall, while the manuscript is readable, improvements in technical clarity, sentence structure, and consistency would elevate the quality of the language. A professional language editing service might also help refine the manuscript further.

Author Response

Dear Reviewer,

Thank you for taking the time to review our work.

We have followed all the suggestions provided. We have highlighted all your comments. We have also written our response below each comment.

Definition of Terms: The manuscript lacks clear definitions for key terms, particularly those related to the n-tuple network and its application in Othello. For instance, the term "network shape" is introduced without sufficient context or clarity, leaving the reader to infer its precise meaning. Similarly, the description of "genetic algorithms" and "tuples" could benefit from more detail to make the paper accessible to a broader audience.

[Response]
We added the following lines to Section 2 and 4 to describe "network shape" and "genetic algorithms".

> Throughout in this paper, the term "tuple shape" refers to this subregion a_i, and "network shape" refers to its set a = {a₁, ..., a_m}.
> The proposed method uses a genetic algorithm (GA).  GA is an optimization method inspired by the theory of evolution, which posits that the fittest individuals are more likely to survive and reproduce. In a GA, multiple individuals, each represented by a chromosome-like data structure, are generated. These individuals are evaluated based on a fitness function that quantifies how well a particular solution solves the given problem. The algorithm proceeds by selecting individuals with higher fitness values, and then applying genetic operators such as crossover and mutation to generate new offspring. This process is repeated iteratively until a satisfactory solution is found.

Explanation of Othello: The paper assumes prior knowledge of Othello's mechanics and strategy. A brief introduction to the game, its rules, and why certain features (e.g., corners, edges) are significant would enhance reader understanding and contextualize the study.

[Response]
We added the following lines to Introduction to describe Othello.

> Othello is a two-player abstract strategy board game in which players alternate between placing black or white disks on an 8x8 board. The objective is to have the majority of one's color disks on the board when the game ends. A disk is placed on a square to places one or more of the opponent's disks in a horizontal, vertical, or diagonal line. Any placed opponent's disks so placed are flipped and become the placing player's color. When a player cannot make a valid move, the turn is passed to the opponent. The game ends when neither player can make a move, typically when the board is full.
> For example, the corner and edge areas are important in Othello, because corner disks are never flipped and edge disks are less likely to be flipped. But stones are rarely placed in these areas early in the game. Therefore, it is not until the middle game that the corner areas represent points of interest.

Mathematical Clarity: The mathematical explanations, such as Equations (1) and (2), are presented without adequate interpretation or examples. This makes it challenging for readers unfamiliar with these concepts to follow the methodology and results. Including more detailed explanations and practical examples would improve clarity.

[Response]
We added Equation (1) example.

> The following is an example of the computation of Equation (1) on the board x represented in Figure 1. For simplicity, the network consists of only one tuple, whose shape a₁ = {56, 57, 58, 59, 60, 61, 62, 63}, highlighted in blue in Figure 1. In this case, x(a₁₁), x(a₁₂), ... , x(a₁₇) are 2, 1, 1, 2, 2, 1, 0, and the ternary notation for ∑_j=1^n_i  3^j-1 x(a_1j) is 2112210, which is the concatenation of these numbers. Therefore, $f(x) = w₁[2112210₍₃₎]

Structure and Presentation: Although the paper has merit, the organization could be improved. For example, Section 5 introduces experiments without fully setting up the theoretical groundwork, and there is limited discussion on the broader implications of the findings.

[Response]
We added the following lines at the beginning of Section 5 to describe the role of each experiment.

> This paper conducts three experiments. In the first experiment, we optimize the network shape for each representative move using the proposed method. To verify the effectiveness of the optimized network shape, we evaluate the prediction precision in the dataset in the second experiment, and the performance in the game in the third experiment.

We added considerations in applying to a wider range of areas at Discussion.

> In such cases, it may be necessary to redesign the fitness function. While this paper defined the fitness function as the prediction accuracy on a dataset, it is also possible to design the fitness function as match performance in reinforcement learning, as demonstrated in Experiment (5.3). This approach, although computationally more expensive, eliminates the need for prior knowledge in the form of a dataset, allowing for broader application to a wider range of problems. In addition, when applied to larger state spaces, such as Go, the network size needs to be larger. Verifying this applicability and scalability is one of our future tasks.

Thank you very much for reviewing our work.

Best regards,

Hiroto Kuramitsu
Graduate School of Science and Technology
Tokyo University of Science