Counting Crowded Soybean Pods Based on Deformable Attention Recursive Feature Pyramid
Round 1
Reviewer 1 Report
The research is a good idea, but according to the researcher, it is still under experiment and has many errors
The research lacked any means of statistical analysis of the data
The method used in counting pods lacked detection of healthy pods from damaged ones
Or calculate the length of the pods and the number of internal seeds in it
So, imaging can be linked to other calculations instead of counting only, such as the length and thickness of the pods, the number of seeds per pod, and the detection of green and ripe pods based on the degree of color.
The search as a whole is good, but it needs more improvement in the programs and algorithms used
But it lacks any statistical comparison tool
Author Response
Point 1: The research lacked any means of statistical analysis of the data.
Response 1: We appreciate the reviewer’s comments and advice.
We had considered some statistical analysis methods for the experimental results but unfortunately failed. In fact, the experimental results are obtained in the entire test set and are given by independent values such as average precision, average recall rate and average F1, etc. These experimental results are relatively stable once the test data and model parameters are determined, which results in conventional statistical analysis methods such as variance, standard deviation, covariance analysis cannot be developed.
Point 2: The method used in counting pods lacked detection of healthy pods from damaged ones Or calculate the length of the pods and the number of internal seeds in it. So, imaging can be linked to other calculations instead of counting only, such as the length and thickness of the pods, the number of seeds per pod, and the detection of green and ripe pods based on the degree of color.
Response 2: We appreciate the reviewer’s comments and advice.
In this work, we focus on recognizing and locating each pod for counting. As you mentioned, further fine-grained measurements of pod phenomics would be of great value. However, the challenges of predicting these phenomics traits on imaging quality and algorithm performance cannot be ignored. For example, for dense small-sized pods, the pixel-level deviation of the predicted foreground regions will lead to huge fluctuations in the pod length prediction results. The thickness of pod requires additional spatial information provided by 3D point clouds or RGB-D images or multi-view RGB images. Based on this work, we provide more discussion about application prospects and extension ideas including the prediction of pod maturity, shape, length, color, the number of seeds per pod and even the number of pod fuzz.
Line 423
>> Combination with more fine-grained pod phenomics. Although pod counting is important for both breeding and cultivation task, considering the combination with other fine-grained pod phenomics measurements shows a more promising future. The method proposed in this paper can accurately identify and locate dense small-sized pods, which can provide instance-level research objects for further detailed analysis of pod length, thickness, shape, color, maturity, and disease conditions. In addition, the pod counting task can also be extended to the prediction of the number of pod seeds [25] or even the number of pod fluff, which is of great significance for the breeding of high-yield and disease-resistant soybean varieties. However, we also note the difficulty in constructing the above multi-task models, especially in terms of imaging quality and algorithm performance. For example, for dense small-sized pods, the pixel-level deviation of the predicted foreground area will lead to huge fluctuations in the length of the pod. The thickness of the pod requires additional spatial information from 3D point clouds or RGB-D images or multi-view RGB images. From the perspective of algorithm design, a conventional solution is to directly construct a regression model by combining the measured data of specific traits, which is cost-effective for pod length and thickness that are easy to measure manually. Meanwhile, the design of multi phenomics algorithm can introduce the ensemble learning, contrastive learning, weakly supervised learning and multi-modal learning. In order to improve the accuracy and generalization of the model, we also try to embed agricultural expert knowledge into the learning of vision tasks. What's more, combined with some related latest research [26], mining the relationship between phenotypic trait results and gene sequences can also be considered.
Reference
[25] Gulzar, Y.; Hamid, Y.; Soomro, A.B.; Alwan, A.A.; Journaux, L. A convolution neural network-based seed classification system. Symmetry 2020, 12, 2018.
[26] Aggarwal, S.; Gupta, S.; Gupta, D.; Gulzar, Y.; Juneja, S.; Alwan, A.A.; Nauman, A. An Artificial Intelligence-Based Stacked Ensemble Approach for Prediction of Protein Subcellular Localization in Confocal Microscopy Images. Sustainability 2023, 15, 1695.
Author Response File: 
Author Response.pdf
Reviewer 2 Report
The author has proposed “Counting Crowded Soybean Pods based on Deformable Attention Recursive Feature Pyramid”. However, I have following comments.
- Details about the number of images in a dataset are missing from abstract.
- Number of classes in the dataset is also missing from abstract.
- Loss rate is also missing from the abstract
- The contribution of the article is missing in the introduction. Author should show the contribution of the manuscript in built points.
- The literature is not sufficient to cover the said area. Many latest references have not been mentioned in the literature. The author should include the latest literature in the manuscript and highlight their contribution. Some of the studies are as follows:
- A convolution neural network-based seed classification system; Fruit Image Classification Model Based on MobileNetV2 with Deep Transfer Learning Technique; Enhancing Image Annotation Technique of Fruit Classification Using a Deep Learning Approach; An Artificial Intelligence-Based Stacked Ensemble Approach for Prediction of Protein Subcellular Localization in Confocal Microscopy Images.
- Table 1: author is using deep learning so they are aware about how much data is important in deep learning, yet from the table it can be seen that they data is very limited like one class is having 48 images for training whereas it can be seen that another class has just 12 images. in this case even data augmentation wont help here. It is must that author should train model on larger dataset such as minimum 100 to 150 images from each class.
- Table 2 should come after 3.2 section. You cant put table/figure before u cite them
- Author must be ablation study to find out how their proposed model has achived highest accuracy than others.
- Author also must do the comparison of their proposed model with the state-of-art models such as mobilenetv2, inceptionv3, YOLOv5, alexnet, vgg19
- There is no practical usage of model mentioned in introduction nor in conclusion
- Future plan is missing
Moderate editing of English language
Author Response
Response to Reviewer 2 Comments
Point 1: Details about the number of images in a dataset are missing from abstract. Number of classes in the dataset is also missing from abstract. Loss rate is also missing from the abstract.
Answer:
Response 1: We appreciate the reviewer’s comments and advice.
As request, we have add more details of the dataset and final loss in the abstract, which can be find from Line 11 to 15.
>> The model is trained on a dense soybean dataset with more than 5300 pods from three different shapes and two classes, which consists of a training set of 138 images, a validation set of 46 images and a test set of 46 images. Extensive experiments have verified the performance of proposed DARFP-SD. The final training loss is 1.281, and an average accuracy of 90.35%, an average recall of 85.59% and a F1 score of 87.90% can be achieved, outperforming the baseline method VFNet 8.36%, 4.55% and 7.81%, respectively.
Point 2: The contribution of the article is missing in the introduction. Author should show the contribution of the manuscript in built points.
Response 2: We appreciate the reviewer’s comments and advice.
As request, we have add our contributions in the introduction point by point, which can be find in Line 88.
>>In summary, our contributions are as follows: (1) A detailed review is conducted to examine the most notable work in soybean pods based on deep learning, and challenges of crowding and uneven distribution in practical applications of pod counting are summarized. (2) A deformable attention recurrent feature pyramid network is specifically designed, which adaptively extracts fine-grained soybean features and assigns suitable NMS threshold to improve the counting performance of crowded and uneven soybean pods. (3) Extensive experiments are conducted on the constructed soybean pods dataset. Quantitative and qualitative results validate the effectiveness of the proposed method, which significantly outperforms baseline methods in different scenarios and can achieve state-of-the-art performance compared to previous counting methods.
Point 3: The literature is not sufficient to cover the said area. Many latest references have not been mentioned in the literature.
Response 3: We appreciate the reviewer’s comments and advice.
We have added more latest references in the introduction and discussion, which can be find in Line 36 and Line 428, respectively.
Line 36>> gulzar et al.[11,12] carry out a series of work focus on the fruits classification based on deep learning.
Line 428>>In addition, the pod counting task can also be extended to the prediction of the number of pod seeds [25] or even the number of pod fluff, which is of great significance for the breeding of high-yield and disease-resistant soybean varieties.
Line 442>>What's more, combined with some related latest research [26], mining the relationship between phenotypic trait results and gene sequences can also be considered.
Point 4: Table 1: author is using deep learning so they are aware about how much data is important in deep learning, yet from the table it can be seen that they data is very limited like one class is having 48 images for training whereas it can be seen that another class has just 12 images. in this case even data augmentation wont help here. It is must that author should train model on larger dataset such as minimum 100 to 150 images from each class.
Response 4: We appreciate the reviewer’s comments and advice.
Data-driven deep learning methods do require training data of sufficient scale. For the experimental dataset constructed in this work, although it has only 230 images, it has labeled more than 5300 instance-level pods from three different soybean plant types. For each category, the bounding boxes involved in the training stage are more than 2000 after data enhancement. Therefore, existing dataset can support the construction of detectors both theoretically and practically. Of course, we also plan to build and open source soybean experimental datasets covering more varieties and multiple growth stages.
Point 5: Table 2 should come after 3.2 section. You cant put table/figure before u cite them.
Response 5: We appreciate the reviewer’s comments and advice.
We have modified the location of Table 2 and Figure 5.
Point 6: Author must be ablation study to find out how their proposed model has achived highest accuracy than others.
Response 6: We appreciate the reviewer’s comments and advice.
For the proposed DARFP-SD, it consists of the module of attention deformable recursive pyramid (ARFP) and bounding box refinement. To quantify the improvement of two modules for our algorithm, we have conducted extensive ablation studies demonstrate the effectiveness of our key components to in Table 3-5.
Point 7: Author also must do the comparison of their proposed model with the state-of-art models such as mobilenetv2, inceptionv3, YOLOv5, alexnet, vgg19.
Response 7: We appreciate the reviewer’s comments and advice.
Apart from YOLOV5, well-designed backbones such as mobilenetv2, inceptionv3, alexnet or vgg19 cannot be directly applied to detection and counting tasks. In order to compare the proposed method with more state-of-art models, we replace the ResNet of our DARFP-SD with the above mentioned four backbones to extract features and report the counting performance in Table 2 in Line 313.
>> We also conduct a set of experiments to study the counting performance with various backbones in Table 2, such as VGG16, AlexNet, DarkNet53 and MobilenetV3. Compared with the well-designed ResNet, the counting results show a slight drop to varying degrees.
Point 8: There is no practical usage of model mentioned in introduction nor in conclusion.
Response 8: We appreciate the reviewer’s comments and advice.
The proposed method aims to serve the automatic pod counting for a single soybean plant, especially in terms of the dense and uneven number distribution scenarios. We have added more description of the practical usage of proposed model in Line 462.
>>We believe the proposed DARFP-SD can give some new insights in the automatic counting task of crop organs, and relieve the manual workload when measuring the pods number per plant during soybean breeding.
Point 9: Future plan is missing
Response 9: We appreciate the reviewer’s comments and advice.
We have added our future plan in the discussion and conclusion in Line 423 and 462.
Line 423>>Combination with more fine-grained pod phenomics. Although pod counting is important for both breeding and cultivation task, considering the combination with other fine-grained pod phenomics measurements shows a more promising future. The method proposed in this paper can accurately identify and locate dense small-sized pods, which can provide instance-level research objects for further detailed analysis of pod length, thickness, shape, color, maturity, and disease conditions. In addition, the pod counting task can also be extended to the prediction of the number of pod seeds [25] or even the number of pod fluff, which is of great significance for the breeding of high-yield and disease-resistant soybean varieties. However, we also note the difficulty in constructing the above multi-task models, especially in terms of imaging quality and algorithm performance. For example, for dense small-sized pods, the pixel-level deviation of the predicted foreground area will lead to huge fluctuations in the length of the pod. The thickness of the pod requires additional spatial information from 3D point clouds or RGB-D images or multi-view RGB images. From the perspective of algorithm design, a conventional solution is to directly construct a regression model by combining the measured data of specific traits, which is cost-effective for pod length and thickness that are easy to measure manually. Meanwhile, the design of multi phenomics algorithm can introduce the ensemble learning, contrastive learning, weakly supervised learning and multi-modal learning. In order to improve the accuracy and generalization of the model, we also try to embed agricultural expert knowledge into the learning of vision tasks. What's more, combined with some related latest research [26], mining the relationship between phenotypic trait results and gene sequences can also be considered.
Line 464>>In the follow-up work, we will build a counting model that integrates more fine-grained phenotypic traits and mine the potential genetic relationship between these traits and gene sequences.
Reference
[11] Mamat, N.; Othman, M.F.; Abdulghafor, R.; Alwan, A.A.; Gulzar, Y. Enhancing Image Annotation Technique of Fruit Classification Using a Deep Learning Approach. Sustainability 2023,15, 901.
[12] Gulzar, Y. Fruit Image Classification Model Based on MobileNetV2 with Deep Transfer Learning Technique. Sustainability 2023, 15, 1906.
[25] Gulzar, Y.; Hamid, Y.; Soomro, A.B.; Alwan, A.A.; Journaux, L. A convolution neural network-based seed classification system. Symmetry 2020, 12, 2018.
[26] Aggarwal, S.; Gupta, S.; Gupta, D.; Gulzar, Y.; Juneja, S.; Alwan, A.A.; Nauman, A. An Artificial Intelligence-Based Stacked Ensemble Approach for Prediction of Protein Subcellular Localization in Confocal Microscopy Images. Sustainability 2023, 15, 1695.
Author Response File: Author Response.pdf
Round 2
Reviewer 1 Report
good paper and good work
Reviewer 2 Report
The author has responded to all the comments and has incorporated the necessary ones.
Moderate editing of English language required
