Improved Cascade Correlation Neural Network Model Based on Group Intelligence Optimization Algorithm
Round 1
Reviewer 1 Report
The authors studied some existing optimization algorithms in the traditional neural network that have various disadvantages of single optimization goal, slow convergence speed, and easy to fall into local, which cannot fully meet the key elements in the cascade correlation learning algorithm.
This paper contains some new results in the field of neural network model based on group intelligence optimization algorithm. This paper needs a major revisions.
1. The authors add an independent paragraph to highlight the manuscript's objectives.
2. The authors must check the captions of tables and figures.
3. Please rewrite examples for more discussion.
4. Please add detailed examples related to the suggested operator.
5. Please discuss the accuracy of simulation results in Table 6.
6. Please update the references [29], [31] and [41].
7. Please add more discussion in Section 4.2. MOEA-T algorithm.
8. Please improve comparison analysis, the group intelligence optimization algorithm can take into account these key elements in the optimization process at the same time, and obtain better optimization results.
9. Literature review should be improved. It does not show research gap. Some recent papers and contributions are missing. You must discuss the literature in the proper sequence while analyzing it. Please read some recent articles.
10. Please add future directions related to use of proposed work in decision making problems.
The authors can add a future plan for further investigation under linear Diophantine fuzzy sets based optimization methods and models.
Author Response
Dear Reviewer,
Thank you very much for your time involved in reviewing the manuscript and your valuable feedback.
Comments:
“The authors studied some existing optimization algorithms in the traditional neural network that have various disadvantages of single optimization goal, slow convergence speed, and easy to fall into local, which cannot fully meet the key elements in the cascade correlation learning algorithm.
This paper contains some new results in the field of neural network model based on group intelligence optimization algorithm. This paper needs a major revisions.
”
Response:
We really appreciate your clear and detailed feedback! Thank you for your constructive advice and we have carefully made amendments in accordance with it. In the remainder of this letter, we thoroughly discuss each of your comments individually along with our corresponding responses.
----------------------------------------------------------------------------------------
Comment 1:
“The authors add an independent paragraph to highlight the manuscript's objectives. ”
Response 1:
Thanks for your suggestions. We have modified as required. Details are as follows:
“In this paper, we present the improved Cascade Correlation neural network model based on single objective group intelligent optimization algorithm jDE-B and the improved Cascade Correlation neural network model based on multi-objective group intelligent optimization algorithm MOEA-T. Compared with the original Cascade Correlation neural network model, their final training results all reduce the required number of hidden units. Among them, the former focuses on reducing the total number of hidden units, while the latter focuses on reducing the network depth.”
“The ultimate goal of this study is to improve the ability of the Cascade Correlation neural network fitting problems, reduce the required number of hidden units and network depth, optimize the network structure, and explore the performance of the group intelligent optimization algorithm in the cascaded neural network model when combined with the correlated neural network algorithm.”
(Modify place：line 140-151)
----------------------------------------------------------------------------------------
Comment 2:
“The authors must check the captions of tables and figures.”
Response 2:
Thanks for your suggestions. We have checked the captions of tables and figures again and made some modifications. Details are as follows:
“Figure 4. Steps of CCNN-MOEA-T algorithm”
“Figure 8. Each algorithm drops the loss value on the four spirals classification problem”
“Figure 11. The multi-objective method fits the output unit’s fluctuations during training”
(Modify place：line 417,line 473, line 537)
----------------------------------------------------------------------------------------
Comment 3:
“Please rewrite examples for more discussion.. ”
Response 3:
Thanks for your suggestions. We have rewrite some examples for more discussion. Details are as follows:
“As can be seen from figure 6, CCNN-jDE-B algorithm performs best which depends on the jDE-B algorithm can better generate hidden unit. The jDE-B algorithm has stronger global search ability and stability.”
“As can be seen from the table 6, the constructed network structure is smaller than the original network structure, but the accuracy is difficult to improve.”
(Modify place：line 454-456, line 596)
----------------------------------------------------------------------------------------
Comment 4:
“Please add detailed examples related to the suggested operator. ”
Response 4:
Thanks for your suggestions. We have added more examples related to some suggested operators. Details are as follows:
The suggested operator in jDE-B algorithm is mainly about the mechanism of self-inspection jumping out of the local area.The suggested operator in MOEA-T algorithm is mainly about the mechanism of propulsive population.
The suggested operation is mainly about the group intelligent optimization method and multi-objective optimization scheme. The detailed examples can be found in two spirals classification problems and four spirals classification problems.
(Modify place：line 375-384)
----------------------------------------------------------------------------------------
Comment 5:
“Please discuss the accuracy of simulation results in Table 6.”
Response 5:
Thanks for your suggestions. We have modified as required. Details are as follows:
“The LeNet-5 network model is limited by the network size and has only achieved about 60% accuracy on the CIFAR-10 datasets.”
“As can be seen from the table 6, the constructed network structure is smaller than the original network structure, but the accuracy is difficult to improve.”
(Modify place：line 590-597)
----------------------------------------------------------------------------------------
Comment 6:
“Please update the references [29], [31] and [41].”
Response 6:
Thanks for pointing this out. We have modified as required. Details are as follows:
“29. Elbisy M S , Ali H M , Abd-Elall M A , et al. The use of feed-forward back propagation and cascade correlation for the neural network prediction of surface water quality parameters. Water Resources, 2014, 41(6):709-718.”
“31. Das S, Suganthan P N . Differential Evolution: A Survey of the State-of-the-Art. IEEE Transactions on Evolutionary Com-putation, 2011, 15(1):4-31.”
“41. Krizhevsky A, Sutskever I, Hinton G E. Imagenet classification with deep convolutional neural networks. Communictaion of the ACM, 2017, 60(6): 84-90.”
(Modify place：line 647-731)
----------------------------------------------------------------------------------------
Comment 7:
“Please add more discussion in Section 4.2. MOEA-T algorithm.”
Response 7:
Thanks for your suggestions. We have modified as required. Details are as follows:
“During the MOEA-T algorithm, the optimization objective of each propulsive population is a sub-objective of the multi-objective optimization problem. The optimization result is mainly affected by the corresponding weight vector. The multi-objective optimization population starts to initialize after the evolution of the propulsive population, and the optimal individuals in the propulsive populations are retained. Most of the optimal individuals in the population are the marginal solutions, which will affect the optimization direction and convergence speed of the remaining individuals. The final evolution results of the population in the MOEA-T algorithm may not be as uniformly distributed as in the MOEA/D algorithm. However, MOEA-T algorithm can provide high-quality edge solutions in a relatively short time.”
(Modify place：line 375-384)
----------------------------------------------------------------------------------------
Comment 8:
“Please improve comparison analysis, the group intelligence optimization algorithm can take into account these key elements in the optimization process at the same time, and obtain better optimization results.”
Response 8:
Thanks for your suggestions. We have modified as required. Details are as follows:
The first key element is the global search ability. The mainly comparison analysis is in the two spirals classification problem. “As can be seen from figure 6, CCNN-jDE-B algorithm performs best which depends on the jDE-B algorithm can better generate hidden unit. The jDE-B algorithm has stronger global search ability and stability.”
The second key element is the optimization target of the hidden unit. The mainly comparison analysis is in the four spirals classification problem.
Details are as follows
“As can be seen from figure 6, CCNN-jDE-B algorithm performs best which depends on the jDE-B algorithm can better generate hidden unit. The jDE-B algorithm has stronger global search ability and stability.”(line 454-456)
“As can be seen from the table 2, the final number of layers required for the improved multi-objective strategy is much lower on the four spirals problem than that based on the single-objective strategy. This shows that the correlation fluctuations fitted by the multi-objective strategy in each layer are better than those trained by a single hidden unit, but at the cost of generally a greater number of hidden units than a single target. ” ( line 475-480)
“As can be seen from the figure 8, when comparing the multi-objective strategy, the more the number of hidden units is added in each layer, the faster the loss value of each layer of the network decreases. But as the number of hidden units increases each time, this speed reaches an upper limit. That is to say, there is an upper limit on the correlation fluctuations simulated by each layer of the strategy, which cannot be fully fitted within one layer.”(line 483-488)
----------------------------------------------------------------------------------------
Comment 9:
“Literature review should be improved. It does not show research gap. Some recent papers and contributions are missing. You must discuss the literature in the proper sequence while analyzing it. Please read some recent articles.”
Response 9:
Thanks for your suggestions. We have added some of the literature contributions in the paper.The sequence of my analysis of the literature is based on the following:
1.During the introduction, the sequence of the literature is mainly around the key elements in the optimization process. The order is not in chronological order.
2.During the algorithm background, the sequence of the literature is mainly around improvement direction and application.
According to my superficial understanding, there are many papers about the improvement of Cascade Correlation learning algorithm int the past 20 years. And the Cascade Correlation neural network were widely applied to practical problems used in past 10 years.
Most of the recent related papers have combined other methods. The methods in different fields have brought some difficulties to my understanding. And Cascade correlation learning algorithm may be not dominant
This paper focuses on using group intelligence optimization algorithms to solve the optimization problems in Cascade Correlation learning algorithm.
Details are as follows:
“The researchers propose many improvement strategies for the Cascade Correlation learning algorithms. For example, the researchers improve the learning process of the Cascade Correlation learning algorithms by changing the connection of the hidden units[26]. The researchers reduce the redundant hidden units and connection weights by changing the calculation method of the hidden unit connection weights which improve the convergence of the network[27].
The Cascade Correlation neural network can be applied to practical problems. On the problem of river staging and river flow prediction, the Cascade Correlation neural network is able to predict the river stage and river flow more accurately[28]. Similarly, the Cascade Correlation neural network show the advantage of high prediction rate and fast training speed in the prediction of surface water quality parameters[29]. Some researchers also applied the Cascade Correlation neural network to stock prediction, which alleviates training slow and overfitting problems[30].”
(Modify place：line 180-192)
----------------------------------------------------------------------------------------
Comment 10:
“Please add future directions related to use of proposed work in decision making problems.
The authors can add a future plan for further investigation under linear Diophantine fuzzy sets based optimization methods and models.”
Response 10:
Thanks for providing improvement direction. We have added the future work. Details are as follows:
“Finally, our future research may be to refer to other effective strategies to improved the Cascade Correlation neural network for solving decision making problems.”
(Modify place：line 639-642)
----------------------------------------------------------------------------------------
Author Response File: 
Author Response.pdf
Reviewer 2 Report
The article presented brings an improved Cascade Correlation neural network model based on single objective group intelligent optimization algorithm and based on multi-objective group intelligent optimization algorithm. The model is further explained in the paper and the algorithm is presented with relevant equations.
The following need changing:
Please put the figure 1 underneath the first paragraph from 2.1. Cascade Correlation learning algorithm
After this paragraph, that starts as: “The specific CCNN-jDE-B algorithm process is as follows:”, please attach Figure 2.
Please attach the Table 2 after this paragraph that starts like this: “As can be seen from the table 2,” (line 449).
Author Response
Dear Reviewer,
Thank you very much for your time involved in reviewing the manuscript and your valuable feedback.
Comments:
“The article presented brings an improved Cascade Correlation neural network model based on single objective group intelligent optimization algorithm and based on multi-objective group intelligent optimization algorithm. The model is further explained in the paper and the algorithm is presented with relevant equations.”
Responses:
Thank you very much for your time involved in reviewing the manuscript and your valuable
feedback. We have revised the manuscript by providing the complete name of a new abbreviation
and carefully proof-read the manuscript to minimize typographical and grammatical errors.
----------------------------------------------------------------------------------------
Comment 1:
“Please put the figure 1 underneath the first paragraph from 2.1. Cascade Correlation learning algorithm. ”
Response 1:
Thanks for pointing this out. We have modified as required.
(Modify place：line 171)
----------------------------------------------------------------------------------------
Comment 2:
“After this paragraph, that starts as: “The specific CCNN-jDE-B algorithm process is as follows:”, please attach Figure 2.”
Response 2:
Thanks for your suggestions. But we found that Figure 2 has been added before.
(Modify place：line 310)
----------------------------------------------------------------------------------------
Comment 3:
“Please attach the Table 2 after this paragraph that starts like this: “As can be seen from the table 2,” (line 449).”
Response 3:
Thanks for your suggestions. We have modified as required.
(Modify place：line 481)
----------------------------------------------------------------------------------------
Author Response File: Author Response.pdf
Reviewer 3 Report
The paper Improved cascade correlation neural network model based on group intelligence optimization algorithm proposes the single-objective optimization algorithm jDE-B and the multi-objective optimization algorithm MOEA-T, and improvement of the network expansion mode in the learning process of Cascade Correlation neural networks.
The achieved results are very good, the quality of the paper is basically good.
There are, however, some smaller issues which must be corrected.
Some examples of such issues see below.
line 51: The first problem is the optimization method for hidden unit weights have insufficient global search capability.
Bad grammar.
line 91: the variables increase
the number or the values of the variables?
lines 116-118: Thus, the optimization direction of the hidden unit is for the error fluctuations of all the output units, and it is only a simple accumulation, which exists target conflicts possibly.
not fully clear sentence, maybe bad grammar?
lines 140-142: Among them, the improved Cascade Correlation neural network model based on single objective population intelligent optimization algorithm adopts the jDE-B algorithm proposed in the paper.
lines 156-160
The two sentences differ only in single vs multi-objective. Consider to reduce the redundancy.
The second one incorrectly states Section 3 instead of Section 4.
line 182: DE algorithms is suitable...
is -> are
Figure 8: Number of Hidden Layer
bad grammar
Figure 9: X axis labels are not visible at the bottom
Figure 11: X axis labels are not visible at the bottom
Figure 15: X axis title is not visible at the bottom
Author Response
Dear Reviewer,
Thank you very much for your time involved in reviewing the manuscript and your very encouraging comments on the merits.
Comments:
“The paper Improved cascade correlation neural network model based on group intelligence optimization algorithm proposes the single-objective optimization algorithm jDE-B and the multi-objective optimization algorithm MOEA-T, and improvement of the network expansion mode in the learning process of Cascade Correlation neural networks.
The achieved results are very good, the quality of the paper is basically good.
There are, however, some smaller issues which must be corrected.”
Responses:
Thank you very much for your time involved in reviewing the manuscript and your valuable feedback. We have revised the manuscript by providing the complete name of a new abbreviation and carefully proof-read the manuscript to minimize typographical and grammatical errors.
----------------------------------------------------------------------------------------
Comment 1:
“line 51: The first problem is the optimization method for hidden unit weights have insufficient global search capability.
Bad grammar. ”
Response 1:
Thanks for pointing this out. We have modified as required. Details are as follows:
“The first problem is that the optimization method of hidden unit weight does not have enough global search capability.”
(Modify place：line 51)
----------------------------------------------------------------------------------------
Comment 2:
“line 91: the variables increase.the number or the values of the variables?”
Response 2:
Thanks for pointing this out. We have modified as required. Details are as follows:
“the number of the values”
(Modify place：line 92)
----------------------------------------------------------------------------------------
Comment 3:
“lines 116-118: Thus, the optimization direction of the hidden unit is for the error fluctuations of all the output units, and it is only a simple accumulation, which exists target conflicts possibly.
not fully clear sentence, maybe bad grammar?”
Response 3:
Thanks for pointing this out. We have modified as required. Details are as follows:
“Thus, the optimization direction of the hidden unit is for the error fluctuations of all the output units.Howerever, simple accumulation is not a good optimization target because there may be target conflicts possibly.”
“simple accumulation” means that “the sum of the absolute value of the correlation between the output of the new unit and the residual signals of all the output units.”
(Modify place：line 118-120)
----------------------------------------------------------------------------------------
Comment 4:
“lines 140-142: Among them, the improved Cascade Correlation neural network model based on single objective population intelligent optimization algorithm adopts the jDE-B algorithm proposed in the paper.
lines 156-160
The two sentences differ only in single vs multi-objective. Consider to reduce the redundancy.
The second one incorrectly states Section 3 instead of Section 4.”
Response 4
Thanks for pointing this out. We have modified as required. Details are as follows:
“In this paper, we present the improved Cascade Correlation neural network model based on single objective group intelligent optimization algorithm jDE-B and the improved Cascade Correlation neural network model based on multi-objective group intelligent optimization algorithm MOEA-T. Compared with the original Cascade Correlation neural network model, their final training results all reduce the required number of hidden units. Among them, the former focuses on reducing the total number of hidden units, while the latter focuses on reducing the network depth.”
(Modify place：line 140-146,line 156)
----------------------------------------------------------------------------------------
Comment 5:
“line 182: DE algorithms is suitable...
is -> are”
Response 5:
Thanks for pointing this out. We have modified as required. Details are as follows:
“DE algorithms are suitable for handling real number optimization problems and have been successfully applied to solve the real life problems.”
(Modify place：line 195)
----------------------------------------------------------------------------------------
Comment 6:
“Figure 8: Number of Hidden Layer
bad grammar”
Response 6:
Thanks for pointing this out. We have modified as required. Details are as follows:
“Hidden Layers”
(Modify place：line 472)
----------------------------------------------------------------------------------------
Comment 7:
“Figure 9: X axis labels are not visible at the bottom”
Response 7:
Thanks for pointing this out. We have modified as required.
(Modify place：line 504)
----------------------------------------------------------------------------------------
Comment 8:
“Figure 11: X axis labels are not visible at the bottom”
Response 8:
Thanks for pointing this out. We have modified as required.
(Modify place：line 537)
----------------------------------------------------------------------------------------
Comment 9:
“Figure 15: X axis title is not visible at the bottom”
Response 9:
Thanks for pointing this out. We have modified as required.
(Modify place：line 601)
----------------------------------------------------------------------------------------
Author Response File: Author Response.pdf
Round 2
Reviewer 1 Report
This paper is revised carefully.
This paper may be accepted.
