Research on Classification Model of Panax notoginseng Taproots Based on Machine Vision Feature Fusion

Round 1

Reviewer 1 Report

The main concern is that the model being employed is rather standard and broad in nature. In addition to plant recognition, the used models can be applied to any other type of plant recognition. As a result, the justification for employing the deep learning model for panax notoginseng taproots is absent from the discussion. 

The feature extraction, representation learning, or feature fusion methods used for Panax notoginseng taproots could be demonstrated to be more suited for this species than the other baselines, for example. 

Example applications include extracting interpretable intermediate representations from panax notoginseng taproots or performing ablation tests on different fusions. 

Lack of current ConvNet models such as ResNet is one of the minor drawbacks.

Author Response

Dear reviewer:

 

Thank you very much for reviewing my article and for your valuable comments and suggestions. Thank you for your hard work. For your comments and suggestions, I have made the following reply. I don’t know if it will satisfy you. If you have any questions, you can give me feedback through the editor. Thank you again.

 

(1) Through review comments and suggestions, I found that the background material on deep learning in the introduction part of my article is not sufficient and can be improved. (Does the introduction provide sufficient background and include all relevant references).

Reply: In the current research status at home and abroad, I have added relevant documents about the deep learning semantic segmentation model U-net, Deeplabv3+ and PSPNet that I used in the article, which enriches the background material of the article. (Please refer to Article 1 Introduction and References [12], [13], [17], [18])

 

(2) Through the review comments and suggestions, I found that the conclusion of my article can be improved. (Does the result support the conclusion?)

Reply: Because there is no reason to use deep learning for the main root of Panax notoginseng in my discussion. Therefore, I added the reason for using deep learning in the discussion section, and added the use of deep learning process and result summary in the conclusion section.

The conclusion is as follows: because deep learning can automatically extract image features layer by layer from images through different convolutional neural networks, and then classify and recognize them through classifiers. In this study, a deep learning model based on semantic segmentation was established. Three semantic segmentation frameworks, PSPnet, U-net, and Deeplabv3+, were selected as the three-to-seven main root classification models with different head numbers. ResNet50, VGG16, and MobileNet were used as three types of convolutional neural networks. In the feature extraction network, the category average pixel accuracy and average interaction ratio are used as evaluation indicators. The results show that the category average pixel accuracy of PSPnet is 77.98, and the average intersection ratio is 88.97%, both of which are the highest. The performance is the best, so the Panax notoginseng main root classification model established by PSPNet is the best. (Please refer to Article 4 Discussion and Article 5 Conclusion (2)) The IRIV-SVM model is optimized by introducing GWO, GA and PSO algorithms. The results show that the optimization effect of the IRIV-GWO-SVM classification model is the best, and the accuracy of the test set is 98.704 %, the optimization effect is increased by 3.334%. (Please refer to Article 5 Conclusion (4))

 

(3) It is proved that the feature extraction used for the main root of Panax notoginseng, the representation method or the feature fusion method is more suitable for this species than other baselines.

Reply: The most intuitive vision between the taproot categories of Panax notoginseng is the shape and size, but because of its shape there are two types: tumor-shaped and cone-shaped. The tumor-shaped image features of the same number of heads are similar to an ellipse and close to a circle, and the cone-shaped image features are similar to a rectangle. Therefore, it is only possible to classify part of the three-seven taproots with different heads only based on the shape and size, but cannot classify the three-seven taproots with the same head number. However, another feature of the tumor shape has many small bumps on the outside, and carefully observe that the texture distribution is irregular and more, and the color is darker. The outside of the cone is smoother, with less texture and even distribution, and the color is lighter. The shape and size feature extraction method (the smallest circumscribed rectangle, the smallest circumscribed circle), the color feature extraction method (color moment), and the texture feature extraction method (gray co-occurrence matrix) were selected for the feature extraction of the main root of Panax notoginseng and passed the BP, The three classification models of ELM and SVM combine different types of fusion features (shape, size (9 features), shape, size and texture (16 features), shape, size and color (33 features), shape, size, texture and Color (40 features)) method to classify.

The results are as follows: (1) When the number of features is shape and size, among the three classification models, the accuracy of the SVM training set is 76.467%, and the accuracy of the prediction set is 75.185%, both of which are the highest. (2) When the number of features is shape, size, and texture, the accuracy of the prediction set of the BP neural network is 14.782% higher than that without texture features, and the accuracy of the prediction set of ELM is 17.222% higher than that without texture features. SVM's The accuracy of the prediction set is 15.222% higher than that without texture features. It can be seen that texture features are very important in the main root classification of Panax notoginseng. Among the three classification models, the accuracy of the training set and prediction set of the SVM is the highest. (3) When the number of features is shape, size, and color, the accuracy of the prediction set of the BP neural network is 23.309% higher than that without color features, and the accuracy of the prediction set of ELM is 36.297% higher than that without texture features. SVM's The accuracy of the prediction set is 16.111% higher than that without the texture feature. It can be seen that the color feature has important value in the classification of the main root of Panax notoginseng. Among the three classification models, the accuracy of the training set and prediction set of SVM is the highest. (4) When the number of features is shape, size, texture, and color, the prediction set accuracy of BP neural network is 1.47% higher than that with color features without texture features, and 10.005% higher than with texture features without color features. The accuracy of ELM's prediction set is 5.135% higher than that when color features are added without texture features, and it is 24.21% higher than when texture features are added without color features. The accuracy of SVM's prediction set is 0.741% higher than that when color features are added without texture features, and 1.63% is higher than that when texture features are added without color features. It is proved that the feature extraction method of Panax notoginseng taproot is applicable and the Panax notoginseng taproot classification model established by fusion features is the best. It shows that the fusion feature is more suitable for Panax notoginseng taproot classification. (Please refer to section 3.1 of the article)

 

(4) Lack of current ConvNet models such as Resnet

Reply: This article selects three semantic segmentation frameworks of PSPnet, U-net and Deeplabv3+ as the three-to-seven main root classification models with different head counts. ResNet50, VGG16, MobileNetV2 three types of convolutional neural networks are used as feature extraction networks. Among them, VGG16 consists of 13 There are three convolutional layers and three fully connected layers. A total of 16 floors. ResNet50 introduces the residual network. There are two basic blocks, namely Identity Block and Conv Block. Identity Block uses 1x1 convolution to reduce dimensionality before 3x3 network structure, and uses 1x1 convolution to increase dimensionality after 3x3 network structure. Compared with the direct use of 3x3 network convolution, the effect is better, the parameters are fewer, the input dimension and the output dimension are the same, and they can be connected in series to deepen the network. The input and output dimensions of Conv Block are different and cannot be directly connected in series. Its function is to change the dimension of the network. In the MobileNetV2 network part, it uses an inverted residual structure, using 1x1 convolution to increase the dimension before the 3x3 network structure, and after the 3x3 network structure, using 1x1 convolution to reduce the dimensionality, expand first, and then compress. The structure shows that the ResNet50 convolutional neural network in the PSPNet model has better performance as a segmentation network for the main roots of different levels of Panax notoginseng, and it is suitable for grading the main roots of Panax notoginseng. (Please refer to section 3.3.2 of the article)

 

Thank you again and look forward to hearing from you.

 

Best regards,

Yinlong Zhu

Author Response File: Author Response.pdf

Reviewer 2 Report

My primary reservation is that the technique is not unique. The same method can also be used for any other type of plant recognition without modification. 

One possible improvement is to determine what interpretable features can be extracted using the existing method or to justify why such a generalized model is preferable for Panax notoginseng taproots. 

 

One minor point of contention is the absence of state-of-the-art CNNs such as ResNet.

Author Response

Dear reviewer:

 

Thank you very much for reviewing my article and for your valuable comments and suggestions. Thank you for your hard work. For your comments and suggestions, I have made the following reply. I don’t know if it will satisfy you. If you have any questions, you can give me feedback through the editor. Thank you again.

 

(1) Through review comments and suggestions, I found that the conclusion of my article can be improved. (Does the result support the conclusion?)

Reply: Because there is no reason to use deep learning for the main root of Panax notoginseng in my discussion. Therefore, I added the reason for using deep learning in the discussion section, and added the use of deep learning process and result summary in the conclusion section. The conclusion is as follows: because deep learning can automatically extract image features layer by layer from images through different convolutional neural networks, and then classify and recognize them through classifiers. In this study, a deep learning model based on semantic segmentation was established. Three semantic segmentation frameworks, PSPnet, U-net, and Deeplabv3+, were selected as the three-to-seven main root classification models with different head numbers. ResNet50, VGG16, and MobileNetV2 were used as three types of convolutional neural networks. In the feature extraction network, the category average pixel accuracy and average interaction ratio are used as evaluation indicators. The results show that the category average pixel accuracy of PSPnet is 77.98, and the average intersection ratio is 88.97%, both of which are the highest. The performance is the best, so the Panax notoginseng main root classification model established by PSPNet is the best. (Please refer to Article 4 Discussion and 5 Conclusion (2)) Introduce GWO, GA and PSO algorithms to optimize the IRIV-SVM model. The results show that the IRIV-GWO-SVM classification model has the best optimization effect, and the test set accuracy rate reaches 98.704% , The optimization effect is increased by 3.334%. (Please refer to Article 5 Conclusion (4))

 

(2) What interpretable features can be extracted using existing methods, or prove why this generalized model is more suitable for the main root of notoginseng?

Reply: This article selects the shape and size feature extraction method (minimum circumscribed rectangle, smallest circumscribed circle), color feature extraction method (color moment), texture feature extraction method (gray co-occurrence matrix) to extract a total of 40 Panax notoginseng main root features. They are (mapping perimeter, mapping area, width and height of the circumscribed rectangle, width and height of the smallest circumscribed rectangle, slender length (height/width of the smallest circumscribed rectangle), duty cycle (the area of ​​the contour area divided by the area of ​​the smallest circumscribed rectangle) ), the radius of the smallest circumcircle, the first moment (mean), second moment (variance), third moment (skewness) and four of R, G, B and H, S, and V Order moment (kurtosis) and Homogeneity, Contrast, Dissimilarity, Entropy, Energy, Correlation, Auto_Correlation Forty types of interpretable features.) And through the three classification models of BP, ELM, SVM, different types of fusion features (shape and size (9 features), shape, size and texture (16 features), shape, size and color (33 features), shape, size, texture and color (40 features)). The results are as follows: (1) When the number of features is shape and size, among the three classification models, the accuracy of the SVM training set is 76.467%, and the accuracy of the prediction set is 75.185%, both of which are the highest. (2) When the number of features is shape, size, and texture, the accuracy of the prediction set of the BP neural network is 14.782% higher than that without texture features, and the accuracy of the prediction set of ELM is 17.222% higher than that without texture features. SVM's The accuracy of the prediction set is 15.222% higher than that without texture features. It can be seen that texture features are very important in the main root classification of Panax notoginseng. Among the three classification models, the accuracy of the training set and prediction set of the SVM is the highest. (3) When the number of features is shape, size, and color, the accuracy of the prediction set of the BP neural network is 23.309% higher than that without color features, and the accuracy of the prediction set of ELM is 36.297% higher than that without texture features. SVM's The accuracy of the prediction set is 16.111% higher than that without the texture feature. It can be seen that the color feature has important value in the classification of the main root of Panax notoginseng. Among the three classification models, the accuracy of the training set and prediction set of SVM is the highest. (4) When the number of features is shape, size, texture, and color, the prediction set accuracy of BP neural network is 1.47% higher than that with color features without texture features, and 10.005% higher than with texture features without color features. The accuracy of ELM's prediction set is 5.135% higher than that when color features are added without texture features, and it is 24.21% higher than when texture features are added without color features. The accuracy of SVM's prediction set is 0.741% higher than that when color features are added without texture features, and 1.63% is higher than that when texture features are added without color features. It is proved that the feature extraction method of Panax notoginseng taproot is applicable and the Panax notoginseng taproot classification model established by fusion features is the best. It shows that the fusion feature is more suitable for Panax notoginseng taproot classification. (Please refer to section 3.1 of the article)

 

(3) Lack of the most advanced CNN such as Resnet

Reply: This article selects three semantic segmentation frameworks of PSPnet, U-net and Deeplabv3+ as the three-to-seven main root classification models with different head counts. ResNet50, VGG16, MobileNetV2 three types of convolutional neural networks are used as feature extraction networks. Among them, VGG16 consists of 13 There are three convolutional layers and three fully connected layers. A total of 16 floors. ResNet50 introduces the residual network. There are two basic blocks, namely Identity Block and Conv Block. Identity Block uses 1x1 convolution to reduce dimensionality before 3x3 network structure, and uses 1x1 convolution to increase dimensionality after 3x3 network structure. Compared with the direct use of 3x3 network convolution, the effect is better, the parameters are fewer, the input dimension and the output dimension are the same, and they can be connected in series to deepen the network. The input and output dimensions of Conv Block are different and cannot be directly connected in series. Its function is to change the dimension of the network. In the MobileNetV2 network part, it uses an inverted residual structure, using 1x1 convolution to increase the dimension before the 3x3 network structure, and after the 3x3 network structure, using 1x1 convolution to reduce the dimensionality, expand first, and then compress. The structure shows that the ResNet50 convolutional neural network in the PSPNet model has better performance as a segmentation network for the main roots of different levels of Panax notoginseng, and it is suitable for grading the main roots of Panax notoginseng. (Please refer to section 3.3.2 of the article)

 

Thank you again and look forward to hearing from you.

 

Best regards,

Yinlong Zhu

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

The feedback included in this edition gave a justification for the use of the proposed models, which is a valuable piece of evidence. As an alternative, I recommend including those justifications in the text together with a highlight and/or a structural description, such as demonstrating issues in the data and the appropriate techniques, or showing the used model can do this job without the proposed feature extraction, but cannot do it well without it.

I believe that the contribution is more on the feature extraction side rather than the classification model side of things. Therefere, I recommend keeping more about feature extractions.

Author Response

Dear reviewer:

 

Thank you very much for reviewing my article and for your valuable comments and suggestions. Thank you for your hard work. For your comments and suggestions, I have made the following reply. I don’t know if it will satisfy you. If you have any questions, you can give me feedback through the editor. Thank you again.

 

(1) Regarding the reasons for using the suggested model in the previous version.

Reply: I have added it to the article according to your suggestion (the article 3.1 section and article 3.3.2 section explain it). The comparison of the results shows that the traditional machine learning model manually selects the feature extraction method than the deep learning model to automatically extract the features, and the feature classification effect after extraction is better. (Explained in section 3.4 of the article)

 

(2) You suggest to keep more information about feature extraction.

Reply: You are right. The most important thing is feature extraction. Only when the most effective features are extracted, can the existing classification model be used to distinguish the best classification effect. I try to keep as much feature extraction information as possible at the same time. The feature information has been redundantly processed and a good classification effect has been obtained. (The article 3.2.1, 3.2.2, 3.3.3 section explains)

 

Thank you again and look forward to hearing from you.

 

Best regards,

Yinlong Zhu

Author Response File: Author Response.pdf

Reviewer 2 Report

This manuscript was reviewed in second time after the following former submission: https://susy.mdpi.com/user/review/review/21336555/UrcInJj5. The authors conducted major revision, and most of the reviewer's concerns are resolved. In my opinion, this manuscript is adequate for publication after minor revision, and I suggest the following comments.

1. What is the main visual differences between the categories? More number of example images for each category and their descriptions will be beneficial to improve the clarification.

2. Captions of sub-figures are not inconsistent with the corresponding sub-figures.

3. There are several typos such as “Auto-correlation” in texts.

Author Response

Dear reviewer:

 

Thank you very much for reviewing my article and for your valuable comments and suggestions. Thank you for your hard work. For your comments and suggestions, I have made the following reply. I don’t know if it will satisfy you. If you have any questions, you can give me feedback through the editor. Thank you again.

 

(1) What are the main visual differences between categories? More example images and descriptions in each category will help improve clarity.

Reply: You are right, more sample images between each category will help improve clarity. The most important visuals between the taproot categories of Panax notoginseng are shape and size, and the subtle differences are texture and color. (Section 3.1 of the article explains through the fusion of different features and the comparison of the results of different classification algorithms.)

 

(2) The title of the sub-picture does not contradict the corresponding sub-picture

Reply: Modified.

 

(3) There are many typos such as "autocorrelation" in the article

Reply: Typos have been changed.

 

Thank you again and look forward to hearing from you.

 

Best regards,

Yinlong Zhu

Author Response File: Author Response.pdf

This manuscript is a resubmission of an earlier submission. The following is a list of the peer review reports and author responses from that submission.

Round 1

Reviewer 1 Report

This paper presents experimental results for classifying Panax notoginseng taproots. The authors extract features and conduct multi-class classification by utilizing classical classifiers such as support vector machine.

Main concern is that the feature extraction and classification methods presented in this manuscript are too outdated. Furthermore, novelty is insufficient to publish in this high-quality journal even though the authors conducted several experiments on real images.

I suggest the following comments to improve the quality of the manuscript.

1. The authors have to emphasize their novelty in the proposed method.

2. The authors need to compare the proposed algorithm with other previous classification algorithms.

3. Panax notoginseng taproots are divided into 9 grades. The authors need to explain why the dataset contains 4 categories and how to define the categories.

4. What is the main visual differences between the categories. Presentation of example images for each category will be beneficial to improve the clarification.

5. Periods are omitted in captions of figures and tables.

6. Sub-captions of Figures are not consistent.

7. There are several typos such as “Auto_correlation” in line 197 and “Auto_Correlation” in line 206.

8. Texts in Figure 8 are not recognizable.

9. There are many omitted blanks and inconsistencies in references.

Reviewer 2 Report

This is an intriguing subject that has the potential to have a significant impact on related areas, such as other plant recognition. 

 

The author's primary strength is that he conducted many experiments, which ensures that the conclusions are sound and believable. Each processing and algorithm is illustrated with an intuitive and logical example. 

However, there is one spot where the author could improve: 

There are numerous AI algorithms, but it is unknown why specific algorithms, such as data preparation and learning, are chosen. There are two possible solutions to this issue: (1) conduct a comprehensive survey of a general algorithm for image or plant recognition and conduct experiments on all of them to demonstrate robust performance under various scenarios (the current set of selected algorithms is far from complete); or (2) focus exclusively on the characteristics of Panax notoginseng taproots and justify the use of algorithms through theory or experiments.

For example, the author may illustrate the extraction of color or texture in classification using ablation tests. While color and texture are unquestionably critical features, it is necessary to justify their exact value, as (1) fewer features may be sufficient for classification and (2) deep learning may completely obviate the need for feature engineering (why bother encoding features by hand). The best case scenario may be that manual features, when used effectively, can reduce the amount of time required to learn (efficiency) and improve the quality of learning (accuracy) compared to deep learning. Otherwise, merely selecting tools from the machine learning toolbox does not make a significant contribution.

Anyway, the author may require additional justification or experimentation to substantiate the detailed benefits.