Review Reports - Case Studies on the Logical Structure of the Algorithms Tabu Search and Threshold Accepting for Generating Solutions in Searching and Solving the Bin-Packing Problem

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

The authors propose a framework to analyze TA and TS algorithms' logical parts in various methodologies and scenarios. My comments are as follow:

It can be seen that this is the extended version of the conference paper: Nájera, V.L., Joaquín Pérez, O., Reyes, L.C., Orta, C.R. and Morales, C.C., 2025, June. Study Cases on Initial Solution and Searching for Tabu Search and Threshold Accepting Algorithms on Bin-Packing Problem. In International Conference on Computational Science and Its Applications (pp. 71-87). Cham: Springer Nature Switzerland. Please ensure that this manuscript is sufficiently extended and presents new developments as compared to the conference version to merit a journal publication.
Please provide content on how the proposed framework would apply to large-scale (high dimensional) problems - e.g., problems with multiple constraints/variables and multiple objectives. Would the conclusions on the methodological interactions still apply?
Please provide analysis on the potential of applying the results of this framework to problems with high levels of nonlinearity/nonconvexity.
Relate the points mentioned above to the reproducibility of this framework. This is because the key value of this work is in the consistency of the proposed framework.
Please provide some content on real-world example(s) that this framework may be applied to - e.g., logistic supply chain, transportation,..etc.

Minor:

Please correct the grammar of the title - e.g., 'Case Studies on the Logical Structure of the...'
Please ensure that the abstract is concise.

Thank you.

Author Response

Comments 1: It can be seen that this is the extended version of the conference paper: Nájera, V.L., Joaquín Pérez, O., Reyes, L.C., Orta, C.R. and Morales, C.C., 2025, June. Study Cases on Initial Solution and Searching for Tabu Search and Threshold Accepting Algorithms on Bin-Packing Problem. In International Conference on Computational Science and Its Applications (pp. 71-87). Cham: Springer Nature Switzerland. Please ensure that this manuscript is sufficiently extended and presents new developments as compared to the conference version to merit a journal publication.

Response 1:

Thank you for pointing this out. This manuscript (new version after working with comments) presents an extended and significantly enhanced version, in scope, depth, and scientific rigor, providing a comprehensive and original contribution to the field of metaheuristic optimization. The above is explained as follows.

A revision of the literature consulted exclusively for Tabu Search-TS and Threshold Accepting -TA algorithms (become successful for solving hard combinatorial optimization problems), shows that some internal logical components of the algorithm are often analyzed independently, other studies have worked in several logical parts at the same time, all of these for improving the algorithm performance, where the researcher’s expertise plays a vital role in enhancing an algorithm's internal logical structure. Despite these advances, there is still a lack of systematic and formal analysis. Most existing studies focus on proposing new algorithmic variants and reporting performance improvements, but they do not isolate, analyze and evaluate the logical structure methodologies that drive these improvements. As a result, many design decisions remain implicit and are difficult to generalize or reproduce, and the practice of empirical expertise continues.

A reflection on the reviewed specialized literature consulted exclusively for TA and TS algorithms shows that up to now, specifically, there has been no work that explores and performs a thorough experimental analysis on the logical aspects of generating the initial solution and neighborhoods to search the problem space. Given their very close relationship within the internal logical structure of the algorithm, it would be interesting to know how both parts impact the performance of the algorithm.

The main contributions of this paper are:

A formal experimental framework is presented to systematically analyze the internal logical structure of the algorithms, design components choices that are traditionally guided by researcher expertise. More specifically, in this paper, isolating, analyzing and evaluating the combined effects of initialization and neighborhood logical parts under controlled conditions of TA and TS algorithms at solving the one-dimensional Bin Packing Problem (BPP).
The experimental results consistently show that simpler internal logical structure of Threshold Accepting and Tabu Search algorithms, specifically probability-guided initialization combined with a single neighborhood operator, can outperform more complex alternatives in general instances of BPP, offering valuable insights for both researchers and practitioners.
This work provides a formal and reproducible framework that validates and systematizes such design principles.

All the above is missing from the original paper and highlighted in the new extended version. So too, it is important to mention other important contributions compared to the earlier version.

A completely rewritten and expanded introduction and state of the art, incorporating recent advances in metaheuristic design.
A detailed and formalized methodological framework, clarifying key structural components of the algorithms and their interactions.
New performance analyses, including boxplot visualization, distribution and dispersion analysis.
Additional statistical validation, incorporating not only the Wilcoxon signed-rank test, but also effect size measures and bootstrap-based confidence intervals.
The inclusion of a linear regression analysis to further interpret performance behavior
New tables and figures supporting a more in-depth analysis.
A comprehensive discussion section, positioning the results within the context of both classical and recent literature.

Fully revised conclusions and references, reflecting the broader scope and impact of the study.

Comments 2: Please provide content on how the proposed framework would apply to large-scale (high dimensional) problems - e.g., problems with multiple constraints/variables and multiple objectives. Would the conclusions on the methodological interactions still apply?

Response 2:

Your comment is very interesting; undoubtedly, the fact that the proposed framework can be applied to high-dimensional problems would increase its value. However, in this article, we validate our contribution only with the one-dimensional Bin-Packing problem, benchmark general instances. This is due to performing a controlled analysis of algorithmic structure without the confounding effects of extreme instance difficulty. Our future work will first focus on extending the principles and approaches of our research to very difficult cases of the same BPP problem variant, thus continuing the analysis of the algorithms' logical structure and introducing new challenges. However, we do plan to apply it to large-scale problems in the future.

Comments 3: Please provide analysis on the potential of applying the results of this framework to problems with high levels of nonlinearity/nonconvexity.

Response 3:

Comments 4: Relate the points mentioned above to the reproducibility of this framework. This is because the key value of this work is in the consistency of the proposed framework.

Response 4:

To facilitate the reproducibility of the results, the proposed workspace is isolated from external factors that could interfere with the research objective. It is to say,

-All algorithmic variants are evaluated under identical parameter settings and stopping criteria, ensuring fair comparison.

-The framework isolates individual components (initialization and neighborhood design), enabling independent replication and analysis.

-The use of publicly available benchmark general instances ensures that results can be reproduced and validated by other researchers.

-The inclusion of multiple statistical analyses (Wilcoxon test, effect size, bootstrap intervals, regression) provides robustness and transparency in the interpretation of results.

The compilers used for algorithm coding, graphical and statistical analysis, and statistical test validation are also mentioned. The computer equipment and operating system used to conduct experimentation for the designed case studies.

We have clarified these points in the Framework, Results Analysis and Discussion sections.

Comments 5: Please provide some content on real-world example(s) that this framework may be applied to - e.g., logistic supply chain, transportation,..etc.

Response 5:

To date, numerous real-world problems can be solved using one-dimensional bin packing problem solution methods. Below are some articles highlighting real-world applications of the bin packing problem. These works are referenced in the introduction section.

a) Munien, C.; Ezugwu, A.E. Metaheuristic algorithms for one-dimensional bin-packing problems: A survey of recent advances and applications. Journal of Intelligent Systems, 2021, 30, 636-663, 1. In this article, the authors highlight several important real-world applications of the Bin-Packing Problem, including cutting industries, transportation, warehousing, and supply chain management.
b) Munien, C.; Mahabeer, S.; Dzitiro, E.; Singh, S.; Zungu, S.; Ezugwu, A.E.S. Metaheuristic approaches for one-dimensional bin packing problem: A comparative performance study. IEEE Access, 2020, 8, 227438-227465. In this article, the authors mention real-world applications of the Bin Packing Problem (BPP) in industrial settings, supply chain management, allocation of memory in computers, and health care.

c) Xu, R.; Romero, S.V.; Tang, J.; Ban, Y.; Chen, X. Digitized counterdiabatic quantum optimization for bin packing problem. EPJ Quantum Technology, 2025, 12, 98, 1. In this article, the authors show an application of the one-dimensional Bin Packing Problem in quantum computing.

Point 1: Please correct the grammar of the title - e.g., 'Case Studies on the Logical Structure of the...'

Response 1: Case Studies on the Logical Structure of the Algorithms Tabu Search and Threshold Accepting for Generating Solutions in Searching and Solving the Bin-Packing Problem.

Reviewer 2 Report

Comments and Suggestions for Authors

The topic of this manuscript is of some interest, particularly because the authors attempt to move the analysis of metaheuristics beyond a “black-box” perspective toward a more fine-grained structural analysis, which is a potentially valuable direction. However, I believe that the current version of the manuscript still suffers from several core methodological problems that directly affect the credibility of the main conclusion, namely that a deterministic initial solution combined with one neighborhood method performs better. Therefore, my current recommendation is major revision.

The authors repeatedly emphasize that no prior work has systematically analyzed the combined effect of initial-solution generation and neighborhood generation, and they build the contribution of this paper on that basis. However, judging from the related-work review and Table 1, this claim seems to rely mainly on the specific TA/TS literature and classification framework selected by the authors, rather than on a sufficiently comprehensive and rigorous body of evidence. Therefore, the novelty claim should be substantially toned down.
The manuscript refers to Algorithm 4 as “GenerateInitialSolution_Deterministic”, yet the pseudocode still contains clearly random steps, such as “Insert randomly an object into a new container” and “Choose randomly a number r between (0,1)”. This indicates that the method is not truly deterministic, but rather closer to a probability-guided random initialization. Similarly, the neighborhood generation procedure also involves random choices of movement, source container, and destination container. If the naming itself is inaccurate, then the conclusions based on the core comparison framework of “random vs. deterministic” are weakened.
In Equation (5), the authors treat ( Y(time) > Y(time) ) or ( Y(quality) > Y(quality }) ) as one of the conditions under which algorithm ( aq ) is considered the winner. According to the definitions given earlier in the manuscript, however, this would imply that a larger time or a larger quality value is preferable, which contradicts the interpretation of the metrics. If this is a formula error, then the analyses based on the dominance region, total scope, and Table 6 all need to be recalculated.
The authors describe the data as “two independent samples”, but then apply a “two-side Wilcoxon signed rank test”. The signed-rank test is normally used for paired samples rather than independent samples; if the samples are truly independent, a Mann–Whitney test or Wilcoxon rank-sum test would be more appropriate. In addition, the manuscript often performs significance testing only after identifying subsets in which one algorithm has already “won”, which introduces selection bias. The paper also lacks effect sizes, exact p-values, and any discussion of multiple-comparison correction.
The selection of the 324 instance subsets was obtained through a TA-based variance procedure, but in the main experiments each instance seems to be represented by only one performance pair, without sufficiently reporting averages, standard deviations, or confidence intervals over repeated runs with multiple random seeds. This is particularly important for metaheuristics involving random initialization and random neighborhood selection. Otherwise, the results may depend heavily on specific random trajectories.
The authors further establish a linear regression of the form ( time = slope \times quality ) for Scenario 1 and force the regression through the origin, arguing that “time = 0, quality = 0”. However, according to the manuscript, quality is defined as ( Q_f / Q_t ), and in practice this quantity cannot be zero. Therefore, the zero-intercept constraint is not well justified. Moreover, time is not actual runtime but rather a count-based metric, which further weakens the physical or algorithmic interpretability of this regression analysis.
The manuscript still contains obvious template residue. For example, Table 5 still includes the sentence “This is a table. Tables should be placed in the main text near to the first time they are cited.” This suggests that the paper has not yet been carefully finalized. In addition, the x-axes in Figures 1–9 are extremely crowded, making detailed comparison difficult. Some statements are also not sufficiently academic in tone; for example, explaining why one scenario performs better by saying that it requires “less source code in the program” is not a rigorous scientific justification. The language and logical flow of the manuscript also require substantial polishing.
The terminology and naming should be made consistent throughout, for example, Threshold Accepting vs. Threshold Acceptance, study cases vs. case studies, and the description of quality versus time.
The authors should also explain more clearly why the tabu tenure in TS was fixed at 7, why the initial temperature in TA was fixed at 1, and why the stopping criterion was uniformly set to 4000 iterations. At present, no parameter-sensitivity analysis is provided.
It would also strengthen the study to include more standard baselines, such as common bin-packing heuristics or more established TA/TS configurations. Otherwise, the current findings read more like a comparison between two author-defined variants than a broadly informative methodological study.
Finally, the figures and tables would benefit from being redesigned as boxplots, distribution plots, or summary statistical tables, rather than line plots showing point-by-point results for all 324 instances.

Author Response

Comments 1: The authors repeatedly emphasize that no prior work has systematically analyzed the combined effect of initial-solution generation and neighborhood generation, and they build the contribution of this paper on that basis. However, judging from the related-work review and Table 1, this claim seems to rely mainly on the specific TA/TS literature and classification framework selected by the authors, rather than on a sufficiently comprehensive and rigorous body of evidence. Therefore, the novelty claim should be substantially toned down.

Response 1: We agree that the original wording may have overstated the novelty of the contribution. In the revised manuscript, we have carefully toned down this assertion to better reflect the scope of our work. Accordingly, the wording of the Introduction section has been modified and structured to delimit the research on the TA and TS algorithms in solving one-dimensional Bin Packing problem. So too for other sections like Discussion and Conclusion.

Comments 2: The manuscript refers to Algorithm 4 as “GenerateInitialSolution_Deterministic”, yet the pseudocode still contains clearly random steps, such as “Insert randomly an object into a new container” and “Choose randomly a number r between (0,1)”. This indicates that the method is not truly deterministic, but rather closer to a probability-guided random initialization. Similarly, the neighborhood generation procedure also involves random choices of movement, source container, and destination container. If the naming itself is inaccurate, then the conclusions based on the core comparison framework of “random vs. deterministic” are weakened.

Response 2:

Thank you very much for your comment, we really appreciate it. We agree that the term “deterministic” is not strictly appropriate, as Algorithm 4 includes stochastic elements such as the random selection of the initial object and the use of a probability-based selection mechanism.

To address this issue, we have revised the terminology throughout the manuscript. Specifically, Algorithm 3 (IS-R) is now referred to as IS-RG and Algorithm 4 (IS-D) is now referred to as IS-PG, the first means an initialization guided randomly and the second means an initialization guided probabilistic. This was modified in all sections. Importantly, even though the first object in the container is randomly selected in both methods, the subsequent construction process differs significantly, leading to structurally different solutions, shown in the experimentation results. This change ensures conceptual accuracy and avoids potential confusion.

Comments 3: In Equation (5), the authors treat ( Y(time) > Y(time) ) or ( Y(quality) > Y(quality }) ) as one of the conditions under which algorithm ( aq ) is considered the winner. According to the definitions given earlier in the manuscript, however, this would imply that a larger time or a larger quality value is preferable, which contradicts the interpretation of the metrics. If this is a formula error, then the analyses based on the dominance region, total scope, and Table 6 all need to be recalculated.

Response 3: Thank you very much for your comment, we really appreciate it. We agree with the comment, effectively there is a writing error in Equation (5), the sign “>” is incorrect, the correct sign is “<”. This was corrected. The results were correctly calculated; it was only a writing error.

Comments 4: The authors describe the data as “two independent samples” but then apply a “two-side Wilcoxon signed rank test”. The signed-rank test is normally used for paired samples rather than independent samples; if the samples are truly independent, a Mann–Whitney test or Wilcoxon rank-sum test would be more appropriate. In addition, the manuscript often performs significance testing only after identifying subsets in which one algorithm has already “won”, which introduces selection bias. The paper also lacks effect sizes, exact p-values, and any discussion of multiple-comparison correction.

Response 4: Thank you very much for your comments, we really appreciate it. We agree with the comments. The first comment was a writing error, because each pair of variants of study cases was evaluated on the same set S containing 324 instances of BPP problem (observations are paired) and the values of performance measures (quality, time) do not assume a normal distribution, therefore, the two-side Wilcoxon signed rank test was applied. This writing error was corrected in the Result Analysis section. For the second error, first applied statistical test, after, beyond statistical significance, the effect size and bootstrap-based confidence interval analyses were performed to assess practical relevance, new tables were added with exact p-values and other statistics of the new analyses, results are revised and commented. So too, a reflection about the results is performed in terms of significant dominance region, identifying scenarios where can be misinterpreted and highlighting those relevant.

Comments 5: The selection of the 324 instance subsets was obtained through a TA-based variance procedure, but in the main experiments each instance seems to be represented by only one performance pair, without sufficiently reporting averages, standard deviations, or confidence intervals over repeated runs with multiple random seeds. This is particularly important for metaheuristics involving random initialization and random neighborhood selection. Otherwise, the results may depend heavily on specific random trajectories.

Response 5: Thank you very much for your comments, we really appreciate it. In the section Framework is described the procedure for the number of runs of algorithm on one instance, it was fixed in 15 per instance. In Result Analysis section it is described that each pair of variants of study cases for two scenarios were executed 15 times on each instance of set S containing 324 instances of BPP problem. The average values of performance measures, quality and time for each one of variants of all studies cases were considered in all analyses.

Comments 6: The authors further establish a linear regression of the form ( time = slope \times quality ) for Scenario 1 and force the regression through the origin, arguing that “time = 0, quality = 0”. However, according to the manuscript, quality is defined as ( Q_f / Q_t ), and in practice this quantity cannot be zero. Therefore, the zero-intercept constraint is not well justified. Moreover, time is not actual runtime but rather a count-based metric, which further weakens the physical or algorithmic interpretability of this regression analysis.

Response 6: Thank you very much for your comments, we really appreciate it. A regression with a zero intercept was assumed because, according to the definition of the variables, if the execution time is zero, the algorithm has neither explored solutions nor generated results; therefore, the quality must also be zero. In this context, forcing the model to pass through the origin ensures consistency between the statistical formulation and the logic of the analyzed process. This description is expanded upon in the Result Analysis section.

Comments 7: The manuscript still contains obvious template residue. For example, Table 5 still includes the sentence “This is a table. Tables should be placed in the main text near to the first time they are cited.” This suggests that the paper has not yet been carefully finalized. In addition, the x-axes in Figures 1–9 are extremely crowded, making detailed comparison difficult. Some statements are also not sufficiently academic in tone; for example, explaining why one scenario performs better by saying that it requires “less source code in the program” is not a rigorous scientific justification. The language and logical flow of the manuscript also require substantial polishing.

Response 7: Thank you very much for your comments, we really appreciate it. All sections of the new version of the article were completely modified. The Result Analysis section was explicitly modified to address this point.

Comments 8: The terminology and naming should be made consistent throughout, for example, Threshold Accepting vs. Threshold Acceptance, study cases vs. case studies, and the description of quality versus time.

Response 8: Thank you very much for your comments, we really appreciate it. All sections of the new version of the article were completely modified to address this point.

Comments 9: The authors should also explain more clearly why the tabu tenure in TS was fixed at 7, why the initial temperature in TA was fixed at 1, and why the stopping criterion was uniformly set to 4000 iterations. At present, no parameter-sensitivity analysis is provided.

Response 9: Thank you very much for your comments, we really appreciate it. However, the primary objective of this study is not to optimize algorithmic performance through parameter tuning, but rather to isolate and analyze the structural impact of initialization and neighborhood strategies. For this reason, all parameters were intentionally fixed across all experimental variants to ensure: Controlled comparison conditions, Reduction of confounding factors, and Reproducibility of results.

The tabu tenure value (7) and the initial temperature (1) were selected as standard baseline values commonly used in the literature for exploratory configurations, Ref. 1, Ref. 2. The stopping criterion of 4000 iterations was chosen to provide a balanced trade-off between computational effort and solution convergence, ensuring that all variants were evaluated under identical computational budgets. It is important to mention that these values, derived from literature, have been used and yielded good results in past experiments.

Also, we emphasize that introducing parameter tuning would add an additional layer of variability, potentially obscuring the specific effects of initialization and neighborhood mechanisms, which are the focus of this work. To clarify this design decision, we have revised the manuscript to explicitly state that:

“This study prioritizes structural analysis over parameter optimization and therefore employs fixed parameter settings across all variants to ensure fair and reproducible comparisons.”

We also acknowledge this as a limitation and identify parameter sensitivity analysis as an important direction for future work.

References

Glover, F. Tabu search—Part I. ORSA J. Comput. 1989, 1, 190–206.
Dueck, G.; Scheuer, T. Threshold accepting: A general purpose optimization algorithm appearing superior to simulated annealing. Journal of computational physics 1990, 90, 161-175, 1.

Comments 10: It would also strengthen the study to include more standard baselines, such as common bin-packing heuristics or more established TA/TS configurations. Otherwise, the current findings read more like a comparison between two author-defined variants than a broadly informative methodological study.

Response 10: Thank you very much for your comments, we really appreciate it. We agree that including additional baselines could provide further context for performance comparison.

However, the primary aim of this study is not to benchmark against state-of-the-art algorithms, but rather to conduct a controlled methodological analysis of specific design logical structures within metaheuristics, namely strategies for the logical parts of initialization and neighborhood.

To achieve this, we deliberately focused on internally consistent algorithmic variants, ensuring that:

-All configurations share the same structural foundation,

-Only the logical parts under study (initialization and neighborhood design) are varied, and

-Observed differences can be directly attributed to these logical parts.

Including external heuristics or highly optimized configurations would introduce additional design differences (e.g., problem-specific tuning, hybridization, adaptive mechanisms), making it difficult to isolate the effects under investigation.

That said, we recognize the value of broader comparisons and have clarified in the revised manuscript that:

The current study is intended as a foundational analysis framework, and Future work will include comparisons with established heuristics and advanced metaheuristic configurations.

Additionally, we have strengthened the discussion to position our findings in relation to existing approaches in literature.

Comments 11: Finally, the figures and tables would benefit from being redesigned as boxplots, distribution plots, or summary statistical tables, rather than line plots showing point-by-point results for all 324 instances.

Response 11: Thank you very much for your comments, we really appreciate it. All sections of the new version of the article were completely modified. The Result Analysis section was explicitly modified to address this point.

Reviewer 3 Report

Comments and Suggestions for Authors

Approximation algorithms, such as Threshold Accepting (TA) and Tabu Search (TS), rely on four core components: parameter tuning, initial solution generation, neighborhood searching, and stopping criteria. Historically, the interaction between initial solution generation and neighborhood searching has lacked comprehensive analysis.

To address this knowledge gap, authors evaluated TA and TS on the One-Dimensional Bin Packing Problem (1D-BPP). Findings revealed that combining a deterministic initial solution with a single neighborhood search method consistently yields high-quality results much faster than employing multiple, intensive search methods. This methodical discovery strongly reinforces previous approaches to optimizing these internal algorithmic components.

The paper is interesting however there are some weakness that should be overcome

Research questions and hypothesis should be clearly stated in the introduction

Why did author selected Bin packing problem as a case study?

Last paragraph in the introduction is not necessary

The literature review is missing the latest important references

The discussion is missing, authors should compare their results to similar studies

The presentation of the results is very long and hard to read and understand

Author Response

Comments 1: Research questions and hypothesis should be clearly stated in the introduction

Response 1: Thank you very much for your comment, we really appreciate it, we have modified the section Introduction to highlight clearly the importance of this research and main contributions

Comments 2: Why did author selected Bin packing problem as a case study?

Response 2:

Thank you very much for your comment, we really appreciate it, we have modified the section Introduction to briefly introduce the one-dimensional Bin Packing problem. This problem was selected for the next reasons.

Benchmark relevance. It is a well-established NP-hard problem with widely available benchmark instances, facilitating reproducibility and comparison.

Controlled complexity. By focusing on general instances (excluding extremely hard cases), the study isolates the impact of algorithmic structure without confounding factors related to extreme instance difficulty.

Generalizability. The insights obtained are not limited to the BPP but are applicable to a broader class of combinatorial optimization problems where similar metaheuristic structures are used.

Structural suitability. The problem naturally incorporates solutions initialization and neighborhood design, which are the key logical parts under study, making it an appropriate testbed for analyzing their effects.

The last reason is pragmatic, since the author’s research group has been working for several years on improving some methods for solving the Bin Packing problem

Comments 3: Last paragraph in the introduction is not necessary

Response 3: Thank you very much for your comment, we really appreciate it, we have modified the section Introduction to highlight clearly the importance of this research and main contributions

Comments 4: The literature review is missing the latest important references

Response 4: Thank you very much for your comment, we really appreciate it, the sections Introduction and Related Works were lightly modified to board this point

Comments 5: The discussion is missing, authors should compare their results to similar studies

Response 5: Thank you very much for your comment, we really appreciate it, we have developed a new comprehensive discussion section, positioning the results within the context of both classical and recent literature

Comments 6: The presentation of the results is very long and hard to read and understand

Response 6: Thank you very much for your comment, we really appreciate it, we have developed a new results analysis section to facilitate reading and understanding