AdaCB: An Adaptive Gradient Method with Convergence Range Bound of Learning Rate
Round 1
Reviewer 1 Report
This work proposes an AdaCB using the concept of boundness in order to improve the training performance for deep learning models. However, the novelty and contribution of the work are poor. The reviewer has the following concerns.
1. What is the definition of "unacceptable learning rate"? The Adabound method already limits the learning rate by using bound functions. Please present the detailed issues of the bound functions proposed in Adabound, in a qualitative manner.
2. LR range test is not the original idea of this paper. The authors in this work just add the LR range test to their algorithm.
3. Clipping method is not the original idea of this paper. The Adabound method already proposes the concept of "gradient clipping" and sophisticated bound functions.
This paper naively tries to mix the existing approaches proposed by its competitors [14] and [18]. Please clarify the novelty of the work, in order to appeal to the readers.
Author Response
Thanks to the reviewer for your valuable comments. we have answered your comments point by point. please check the file attached.
Author Response File: 
Author Response.docx
Reviewer 2 Report
The authors propose a new optimizer called AdaCB for training deep learning models. The proposed approach is verified experimentally, compared with state-of-art methods such as AdaBound, Adam, and SGD (M), showing improved performance.
The paper is well written, and the results are interesting. I have some minor comments that should be incorporated into the paper, prior to considering its publication. The parameters should be written using an italic style. It is not done for all parameters and in all equations. Please, check the whole paper and correct it. Also, the results of the proposed approach should be emphasized by using, e.g. bold style in the tables and by adding “proposed” in the legends of the figures (or using some other way that authors find more appropriate).
Author Response
Thanks to the reviewer for your valuable comments. we have answered your comments point by point, please check the file attached.
Author Response File: Author Response.docx
Reviewer 3 Report
Numerical results should support conclusions in the abstract. In the introduction section, the study's problem statement is not clearly identified. An organization section that describes the upcoming areas of the article must be added at the end of the introduction section. All the related mathematical equations should be stated, and the associated variables must be identified. (Please check carefully). I can see that the proposed method requires numerous inputs. This is undesirable because it will be challenging to implement and tune properly. How do the authors justify that? High-quality figures must be used in the article file. Some equations are stated without a reference. The authors should explain where or how did they obtain such equations. Although the proposed methodology has advantages, the authors should also state its shortcomings. The authors should compare their methodology with other contemporary methodologies and highlight why and how their method is better than the existing ones. The conclusion must reflect the results.Some linguistic errors are discovered. (please check carefully)
The authors should use more 2021 and 2022 references. The future works are not clearly identified.
Author Response
Thanks to the reviewer for your valuable comments. we have answered your comments point by point, please check the file attached.
Author Response File: Author Response.docx
Round 2
Reviewer 3 Report
I have no further comments regarding his article.
