Distinguishing Cyst Nematode Species Using Image Textures and Artificial Neural Networks
Round 1
Reviewer 1 Report
I think there is a lot of improvement required in the neural network design area. There are 2172 texture features and only one layer can classify them, looks very strange to me.
They used built-in MATLAB and WEKA tools, but seems like, they don't have much knowledge about it. Some help from an AI guy is required. There is no term like wide or narrow or medium neural network architecture.
The commonly used terms are deep neural network, shallow network etc. Visualization techniques should be used for confusion matrix and other parameters.
Minor Language editing is required.
Author Response
Reviewer 1
I think there is a lot of improvement required in the neural network design area. There are 2172 texture features and only one layer can classify them, looks very strange to me.
Response: Thank you very much for your careful reading of the manuscript and comments. The manuscript was significantly improved.
There were 2172 texture features before attribute selection. Then the attribute selection was performed. More details have been added as follows:
“We performed the attribute selection using the Best First and CFS (Correlation-based Feature Selection) subset evaluator and selected thirty texture parameters so that the ratio of the number of image textures (30) to cases (300) was 1:10. The set of selected image parameters included 7 textures from color channel B, 7 textures from color channel Z, 4 textures from color channel U, 3 textures from color channel L, 2 textures from color channel G, 2 textures from color channel X, 2 textures from color channel Y, 1 texture from color channel b, 1 texture from color channel R, and 1 texture of images in color channel S (Table 1).”
The table presenting the selected image parameters was added as follows:
Table 1. The selected image textures parameters used to build artificial neural network models to classify accuracies cyst nematode species
|
Color channel |
Selected image textures |
|
L |
LHMean LHPerc90 LS5SZ3SumAverg |
|
b |
bS5SZ3SumAverg |
|
X |
XHMean XHPerc90 |
|
Y |
YHMean YHDomn10 |
|
Z |
ZHMean ZHVariance ZHPerc50 ZHPerc99 ZS5SH3SumVarnc ZS5SZ3SumAverg ZS4RVGLevNonU |
|
R |
RS5SV3DifVarnc |
|
G |
GHMean GHVariance |
|
B |
BHMean BHVariance BHPerc50 BHPerc90 BHDomn10 BS4RHGLevNonU BS4RVFraction |
|
U |
UHMaxm10 US5SN3AngScMom US5SN5SumAverg UATeta2 |
|
S |
SS5SN5Contrast |
They used built-in MATLAB and WEKA tools, but seems like, they don't have much knowledge about it. Some help from an AI guy is required. There is no term like wide or narrow or medium neural network architecture. The commonly used terms are deep neural network, shallow network etc. Visualization techniques should be used for confusion matrix and other parameters.
Response: We totally agree with the Reviewer about deep neural network and shallow neural network. However Narrow Neural Network, Medium Neural Network, and Wide Neural Network were classifier types used in MATLAB and the type depended on first layer size. The classifier types of Bilayered Neural Network and Trilayered Neural Network depended on the number of layers.
It has been specified as follows:
“The neural network models with the specified parameters developed using MATLAB and WEKA are presented in Table 2. We chose the classifiers providing the highest accuracies to be presented in this paper. The classifier types of Narrow Neural Network, Medium Neural Network, and Wide Neural Network used in MATLAB depended on the first layer size. The classifier types of Bilayered Neural Network and Trilayered Neural Network depended on the number of layers. The exemplary neural network architecture is presented in Figure 3.”
Figure 3. The exemplary neural network architecture.
The neural network models with the specified parameters developed using MATLAB and WEKA are presented in Table 2.
Table 2. The parameters of classifiers used to build models using MATLAB and WEKA
|
Application |
Classifier type |
Parameters |
|
MATLAB |
Narrow Neural Network |
first layer size: 10; number of fully connected layers: 1; activation: ReLU; iteration limit: 1000; standardize data: yes; regularization strength (Lambda): 0 |
|
Medium Neural Network |
first layer size: 25; number of fully connected layers: 1; activation: ReLU; iteration limit: 1000; standardize data: yes; regularization strength (Lambda): 0 |
|
|
Wide Neural Network |
first layer size: 100; number of fully connected layers: 1; activation: ReLU; iteration limit: 1000; standardize data: yes; regularization strength (Lambda): 0 |
|
|
Bilayered Neural Network |
first layer size: 10; second layer size: 10; number of fully connected layers: 2; activation: ReLU; iteration limit: 1000; standardize data: yes; regularization strength (Lambda): 0 |
|
|
Trilayered Neural Network |
first layer size: 10; second layer size: 10; third layer size: 10; number of fully connected layers: 3; activation: ReLU; iteration limit: 1000; standardize data: yes; regularization strength (Lambda): 0 |
|
|
WEKA |
WiSARD |
batchSize: 100; bitNo: 8; bleachConfidence: 0.01; bleachFlag: False; bleachStep: 1.0; debug: False; doNotCheckCapabilities: False; mapType: RANDOM; seed: -1; ticNo: 256 |
|
Multilayer Perceptron |
batchSize: 100; debug: False; doNotCheckCapabilities: False; decay: hiddenLayers: a; learningRate: 0.3; momenetum: 0.2; nominalToBinaryFilter:True; normalizeAttributes: True; normalizeNumericClass: True; reset: True; resume: False; seed: 0; trainingTime: 500; validationTreshold: 20 |
|
|
RBF (Radial basis function) Network |
batchSize: 100; clusteringSeed: 1; debug: False; doNotCheckCapabilities: False; maxIts: -1; minStdDev: 0.1; numClusters: 2; ridge: 1.0E-8 |
Equations for the performance metrics have added to the manuscript as follows:
“We used the following Equations 1-9:
(1)
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
where TP: true positive; TN: true negative; FP: false positive; FN: false negative.”
Minor Language editing is required.
Response: The whole manuscript has been checked and English language has been corrected.
Please see the attachment
Author Response File: 
Author Response.pdf
Reviewer 2 Report
The authors proposed a neural network based work for Distinguishing cyst nematode species using image textures and artificial neural networks
The abstract is well written however, the novelty or the methodology of the proposed model is not highlighted.
Some sample images of the "Cyst nematodes" and their types can be shown along with the description in the introduction section.
The related works section is very weak. Authors should enhance this section with more recent literature.
Sample images from the dataset should be shown. The details of the dataset, the number of train and test images, etc should be shown in a table.
Figure 1 should be enhanced. Keep less text and more pictorial representation.
The major drawback of this manuscript is the lack of novelty. The authors have utilized some standard models however there is no discussion on the highlights of the models.
The hyperparameter tuning details should be presented in tables.
Neural network architecture can be shown as an image.
All the performance metrics should be discussed with their equations.
There is no comparison of the obtained results with the recent literature.
Minor editing of English language required
Author Response
Reviewer 2
The authors proposed a neural network based work for Distinguishing cyst nematode species using image textures and artificial neural networks
Response: Thank you very much for your careful reading of the manuscript and comments.
The abstract is well written however, the novelty or the methodology of the proposed model is not highlighted.
Response: It has been corrected as follows:
“The application of parameters selected from a set of 2172 textures of images in color channels L, a, b, X, Y, Z, R, G, B, V, U, and S to build classification models using a Narrow Neural Network, Medium Neural Network, Wide Neural Network, Trilayered Neural Network, WiSARD, Multilayer Perceptron, and RBF Network is a great novelty of the present study.”
Some sample images of the "Cyst nematodes" and their types can be shown along with the description in the introduction section.
Response: The manuscript has been supplemented. Sample images of the cyst nematodes are presented in Figure 2.
The related works section is very weak. Authors should enhance this section with more recent literature.
Response: More recent references have been added. The Introduction has been supplemented with the following statements:
“Cyst nematodes can impair the growth and deform tubers of potatoes [16]. ... A single nematode transcriptomic profiling with long-read sequencing can be used to reannotate the cyst nematode genome [18]. The analysis of the population genetic structure is characterized by the important role in the determination of the variability in the nematode–plant interaction [19].
In addition to the use of destructive methods to evaluate nematode cysts, objective and non-destructive evaluation can be performed using imaging. Computer vision is characterized by better repeatability, a formal quantification of errors of measurements, automation of species identification by computers, and elimination of the effort of human experts [1]. Due to image processing and pattern recognition, machine vision enables the quantitative analysis of qualitative criteria [20]. The application of image analysis can result in the determination of many textural and geometric parameters of images [21]. The image features can be analyzed using artificial intelligence. An approach combining image analysis and traditional machine learning or deep learning was successfully used in previous studies for classification [22-25].”
- Jindo, K.; Teklu, M.G.; van Boheeman, K.; Njehia, N.S.; Narabu, T.; Kempenaar, C.; Molendijk, L.P.G.; Schepel, E.; Been, T.H. Unmanned Aerial Vehicle (UAV) for Detection and Prediction of Damage Caused by Potato Cyst Nematode G. pallida on Selected Potato Cultivars. Remote Sens. 2023, 15, 1429.
- Ste-Croix, D.T.; Bélanger, R.R.; Mimee, B. Single Nematode Transcriptomic Analysis, Using Long-Read Technology, Reveals Two Novel Virulence Gene Candidates in the Soybean Cyst Nematode, Heterodera glycines. Int. J. Mol. Sci. 2023, 24, 9440.
- Nuaima, R.H.; Heuer, H. Genetic Variation among Heterodera schachtii Populations Coincided with Differences in Invasion and Propagation in Roots of a Set of Cruciferous Plants. Int. J. Mol. Sci. 2023, 24, 6848.
- Noutfia, Y.; Ropelewska, E. Innovative Models Built Based on Image Textures Using Traditional Machine Learning Algorithms for Distinguishing Different Varieties of Moroccan Date Palm Fruit (Phoenix dactylifera L.). Agriculture 2023, 13, 26.
- Noutfia, Y.; Ropelewska, E. Comprehensive Characterization of Date Palm Fruit ‘Mejhoul’ (Phoenix dactylifera L.) Using Image Analysis and Quality Attribute Measurements. Agriculture 2023, 13, 74.
- Ropelewska, E. Diversity of Plum Stones Based on Image Texture Parameters and Machine Learning Algorithms. Agronomy 2022, 12, 762.
- Ropelewska, E. Application of Imaging and Artificial Intelligence for Quality Monitoring of Stored Black Currant (Ribes nigrum L.). Foods 2022, 11, 3589.
- Ropelewska, E. Assessment of the Influence of Storage Conditions and Time on Red Currants (Ribes rubrum L.) Using Image Processing and Traditional Machine Learning. Agriculture 2022, 12, 1730.
- Unlersen, M.F.; Sonmez, M.E.; Aslan, M.F.; Demir, B.; Aydin, N.; Sabanci, K.; Ropelewska E. CNN–SVM hybrid model for varietal classification of wheat based on bulk samples. Eur Food Res Technol 2022, 248, 2043–2052.
Sample images from the dataset should be shown.
Response: Sample images from the dataset have been added in Figure 2.
Figure 2. The sample images of cyst nematodes acquired using a digital scanner.
The details of the dataset, the number of train and test images, etc should be shown in a table.
Response: It has been described in more detail as follows:
“Fifty cysts belonging to one nematode species were in one image. We acquired two images for each species. Then, we divided each image into fifty images including one cyst nematode each. The sample images are shown in Figure 2. In total, we obtained images of one hundred cysts of each nematode species. For a dataset including images of three hundred cysts belonging to three nematode species, we performed the image segmentation, ROI determination, and image texture computation separately for each image including one cyst nematode.”
“We performed the attribute selection using the Best First and CFS (Correlation-based Feature Selection) subset evaluator and selected thirty texture parameters so that the ratio of the number of image textures (30) to cases (300) was 1:10. The set of selected image parameters included 7 textures from color channel B, 7 textures from color channel Z, 4 textures from color channel U, 3 textures from color channel L, 2 textures from color channel G, 2 textures from color channel X, 2 textures from color channel Y, 1 texture from color channel b, 1 texture from color channel R, and 1 texture of images in color channel S (Table 1). We performed the classification using a 10-fold cross-validation test mode applying a random division of the image texture dataset into ten parts and treating each part in turn as the test set, and the remaining nine parts of a dataset as the training sets. We performed the learning ten times on different training sets and estimated the overall error as the average of ten errors.”
The table presenting the selected image parameters was added as follows:
Table 1. The selected image textures parameters used to build artificial neural network models to classify accuracies cyst nematode species
|
Color channel |
Selected image textures |
|
L |
LHMean LHPerc90 LS5SZ3SumAverg |
|
b |
bS5SZ3SumAverg |
|
X |
XHMean XHPerc90 |
|
Y |
YHMean YHDomn10 |
|
Z |
ZHMean ZHVariance ZHPerc50 ZHPerc99 ZS5SH3SumVarnc ZS5SZ3SumAverg ZS4RVGLevNonU |
|
R |
RS5SV3DifVarnc |
|
G |
GHMean GHVariance |
|
B |
BHMean BHVariance BHPerc50 BHPerc90 BHDomn10 BS4RHGLevNonU BS4RVFraction |
|
U |
UHMaxm10 US5SN3AngScMom US5SN5SumAverg UATeta2 |
|
S |
SS5SN5Contrast |
Figure 1 should be enhanced. Keep less text and more pictorial representation.
Response: Figure 1 has been corrected.
The major drawback of this manuscript is the lack of novelty. The authors have utilized some standard models however there is no discussion on the highlights of the models.
Response: The novelty of the present study has been emphasized in Abstract and sections 1. Introduction and 4. Conclusions.
Abstract “The application of parameters selected from a set of 2172 textures of images in color channels L, a, b, X, Y, Z, R, G, B, V, U, and S to build classification models using a Narrow Neural Network, Medium Neural Network, Wide Neural Network, Trilayered Neural Network, WiSARD, Multilayer Perceptron, and RBF Network is a great novelty of the present study.”
- Introduction
“This study was aimed at classifying cyst nematodes of Globodera pallida, Globodera rostochiensis, and Heterodera schachtii based on texture parameters using artificial neural networks (ANNs). The application of features selected from 2172 texture parameters of images in color channels L, a, b, X, Y, Z, R, G, B, V, U, and S to build classification models is a novelty of the present study. Due to involving the combination of image processing and artificial intelligence, the discrimination of cyst nematode species was performed in a non-destructive, rapid, effective, and objective manner.”
- Results and Discussion
“Our study revealed that the combination of image processing and machine learning was useful for distinguishing cyst nematode species. The applied approach is innovative in distinguishing cyst nematode species. However, slightly different applications of machine learning and imaging are present in the literature.”
“Our research is innovative compared to research reported in the available literature. The applied approach combining color image processing and artificial neural networks is a new direction of research in distinguishing cyst nematode species. The developed procedure can be used in further studies for the examination of other species of cyst nematodes. Furthermore, research can be expanded with the use of deep learning.”
- Conclusions
“The performed research is innovative. The application of texture parameters extracted from images in color channels L, a, b, X, Y, Z, R, G, B, V, U, and S to build classification models is a great novelty. Therefore, the present study set a new research direction in distinguishing cyst nematode species involving color image processing and artificial neural networks.”
The hyperparameter tuning details should be presented in tables.
Response: The parameter details are presented in Table 2.
Table 2. The parameters of classifiers used to build models using MATLAB and WEKA
|
Application |
Classifier type |
Parameters |
|
MATLAB |
Narrow Neural Network |
first layer size: 10; number of fully connected layers: 1; activation: ReLU; iteration limit: 1000; standardize data: yes; regularization strength (Lambda): 0 |
|
Medium Neural Network |
first layer size: 25; number of fully connected layers: 1; activation: ReLU; iteration limit: 1000; standardize data: yes; regularization strength (Lambda): 0 |
|
|
Wide Neural Network |
first layer size: 100; number of fully connected layers: 1; activation: ReLU; iteration limit: 1000; standardize data: yes; regularization strength (Lambda): 0 |
|
|
Bilayered Neural Network |
first layer size: 10; second layer size: 10; number of fully connected layers: 2; activation: ReLU; iteration limit: 1000; standardize data: yes; regularization strength (Lambda): 0 |
|
|
Trilayered Neural Network |
first layer size: 10; second layer size: 10; third layer size: 10; number of fully connected layers: 3; activation: ReLU; iteration limit: 1000; standardize data: yes; regularization strength (Lambda): 0 |
|
|
WEKA |
WiSARD |
batchSize: 100; bitNo: 8; bleachConfidence: 0.01; bleachFlag: False; bleachStep: 1.0; debug: False; doNotCheckCapabilities: False; mapType: RANDOM; seed: -1; ticNo: 256 |
|
Multilayer Perceptron |
batchSize: 100; debug: False; doNotCheckCapabilities: False; decay: hiddenLayers: a; learningRate: 0.3; momenetum: 0.2; nominalToBinaryFilter:True; normalizeAttributes: True; normalizeNumericClass: True; reset: True; resume: False; seed: 0; trainingTime: 500; validationTreshold: 20 |
|
|
RBF (Radial basis function) Network |
batchSize: 100; clusteringSeed: 1; debug: False; doNotCheckCapabilities: False; maxIts: -1; minStdDev: 0.1; numClusters: 2; ridge: 1.0E-8 |
Neural network architecture can be shown as an image.
Response: It has been presented in Figure 3.
Figure 3. The exemplary neural network architecture.
All the performance metrics should be discussed with their equations.
Response: Equations for the performance metrics have added to the manuscript as follows:
“We used the following Equations 1-9:
(1)
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
where TP: true positive; TN: true negative; FP: false positive; FN: false negative.”
There is no comparison of the obtained results with the recent literature.
Response: The discussion has been expanded. The obtained results have been compared with the literature data, as follows:
“Our study revealed that the combination of image processing and machine learning was useful for distinguishing cyst nematode species. The applied approach is innovative in distinguishing cyst nematode species. However, slightly different applications of machine learning and imaging are present in the literature. In the previous studies performed by Vlaar et al. [13], machine learning and metabolomics were used for the identification of a cyst nematode hatching factor. Very high Pearson’s correlation coefficients reaching 0.89 between metabolite features present in the root exudate, such as a compound of molecular weight 526.17 and solanoeclepin A and hitching were observed [13]. Microscopic images combined with deep learning were applied to discern and count nematode eggs. The average detection accuracy of 97.00%, including algorithm count, human count, and error margin was determined [35]. Whereas deep learning and computer vision were used for the successful identification of species of Globodera quarantine nematodes, such as G. pallida and G. rostochiensis. The accuracy of distinguishing both species reaching 0.88 was found [1]. Kranse et al. [36] reported that a cost-effective and easy-to-build imaging device to acquire images of the root system of a host plant infected with H. schachtii can replace costly microscopy equipment and the combination with machine learning can increase screening speed. The authors [36] found a correlation between automatic and manual counts of the area reaching 0.83 and 0.44 for the nematode females and nematode cysts, respectively. Furthermore, machine learning techniques and remote multispectral reflectance sensors were used to identify nematode damage on a soybean. The accuracy of the classification of asymptomatic and nematode-symptomatic soybean plants reached 0.71 [37]. Remote sensing was also used for the detection of plant stress induced by H. schachtii in sugar beet fields with a 100% discrimination between affected with beet cyst nematode (BCN) or Rhizoctonia crown and root rot (RCRR) and healthy plants [38] and by soybean cyst nematodes with the correlation of 0.58 between initial soybean cyst nematode population density and satellite image intensities [39]. Whereas Lu et al. [40] used fluorescence imaging to count soybean cyst nematodes and in comparison with the microscope counting, the imaging system was about 60% faster. Baretto et al. [41] applied hyperspectral imaging and machine learning to determine symptoms caused by Rizoctonia solani in sugar beet and the scoring of disease incidence was up to 5 times higher than the human visual rating. Our research is innovative compared to research reported in the available literature. The applied approach combining color image processing and artificial neural networks is a new direction of research in distinguishing cyst nematode species. The developed procedure can be used in further studies for the examination of other species of cyst nematodes. Furthermore, research can be expanded with the use of deep learning.”
Minor editing of English language required
Response: The whole manuscript has been checked and English language has been corrected.
Please see the attachment.
Author Response File: Author Response.pdf
Reviewer 3 Report
General comments:
The current results only show the performance of the models on the training set. A training/testing dataset split is needed at the beginning of the experiment. Results on the test set should be reported to evaluate the performance and generalizability of the ML models.
More specific comments:
1. Sentence from Line 42-44 is hard to understand.
2. Please rewrite sentence from Line 89 to 92 to increase readability.
3. Line 130: are these two images containing two different sets of 50 samples or the same set of samples?
4. Line 156: can the authors comment on why these five neural networks were selected and tested?
5. Have the authors tried a hierarchical classification model with 2 binary classifiers: first classifier to make a decision whether the cyst is H. schachtii or not, if not, then move to the second binary classifier which distinguishes between G. pallida and G. rostochiensis?
More proof-reads are needed to increase readability.
Author Response
Reviewer 3
General comments:
The current results only show the performance of the models on the training set. A training/testing dataset split is needed at the beginning of the experiment. Results on the test set should be reported to evaluate the performance and generalizability of the ML models.
Response: Thank you very much for your careful reading of the manuscript and comments.
The classification (testing) was performed for independent datasets. Different datasets were used for training and for testing. The final results are reported in the paper for the test set. It has been supplemented as follows:
“We performed the classification using a 10-fold cross-validation test mode applying a random division of the image texture dataset into ten parts and treating each part in turn as the test set, and the remaining nine parts of a dataset as the training sets. We performed the learning ten times on different training sets and estimated the overall error as the average of ten errors. “
More specific comments:
- Sentence from Line 42-44 is hard to understand.
Response: It has been corrected as follows:
“Cyst nematodes, such as Globodera spp. and Heterodera spp. are obligate endoparasites of roots [3]. For example, G. rostochiensis and G. pallida are parasites of Solanaceae plants. They infect mainly potato, tomato, eggplant, and other Solanum spp. [4].”
- Please rewrite sentence from Line 89 to 92 to increase readability.
Response: It has been corrected as follows:
“Computer vision is characterized by better repeatability, a formal quantification of errors of measurements, automation of species identification by computers, and elimination of the effort of human experts [1]. ”
- Line 130: are these two images containing two different sets of 50 samples or the same set of samples?
Response: It has been specified as follows:
“Fifty cysts belonging to one nematode species were in one image. We acquired two images for each species. Then, we divided each image into fifty images including one cyst nematode each. The sample images are shown in Figure 2. In total, we obtained images of one hundred cysts of each nematode species. For a dataset including images of three hundred cysts belonging to three nematode species, we performed the image segmentation, ROI determination, and image texture computation separately for each image including one cyst nematode.”
- Line 156: can the authors comment on why these five neural networks were selected and tested?
Response: It has been specified as follows:
“The neural network models with the specified parameters developed using MATLAB and WEKA are presented in Table 2. We chose the classifiers providing the highest accuracies to be presented in this paper. The classifier types of Narrow Neural Network, Medium Neural Network, and Wide Neural Network used in MATLAB depended on the first layer size. The classifier types of Bilayered Neural Network and Trilayered Neural Network depended on the number of layers. The exemplary neural network architecture is presented in Figure 3.”
- Have the authors tried a hierarchical classification model with 2 binary classifiers: first classifier to make a decision whether the cyst is H. schachtii or not, if not, then move to the second binary classifier which distinguishes between G. pallida and G. rostochiensis?
Response: The additional classification models have been developed to distinguish Heterodera species from Globodera species. Then Globodera pallida were distinguished from Globodera rostochiensis.
“In the nest step of the analysis, algorithms providing the highest accuracies of distinguishing all three cyst nematode species were used build further classification models. Firstly, cyst nematodes belonging to Heterodera and Globodera genera were distinguished (Tables 7-8). Then, models were developed to classify species of G. pallida and G. rostochiensis belonging to the genus Globodera (Tables 9-10). Cyst nematodes of Heterodera spp. and Globodera spp. were correctly classified in 99.00% for the model built using Multilayer Perceptron and 98.50% for WiSARD and RBF Network (Table 7). In the case of a model built using Multilayer Perceptron, the highest Kappa statistic of 0.9849 was found. The values reaching 1.000 were observed in the case of TPR for Globodera spp. and Precision for Heterodera spp. The lowest FPR of 0.000 was determined for Heterodera spp. (Table 8).
Table 7. The results of the classification of cyst nematodes of Heterodera and Globodera spp. based on models built using image texture parameters
|
Algorithm |
True class |
Predicted class (%) |
Average accuracy (%) |
|
|
Heterodera spp. |
Globodera spp. |
|||
|
WiSARD |
Heterodera spp. |
98 |
2 |
98.50 |
|
Globodera spp. |
1 |
99 |
||
|
Multilayer Perceptron |
Heterodera spp. |
98 |
2 |
99.00 |
|
Globodera spp. |
0 |
100 |
||
|
RBF Network |
Heterodera spp. |
98 |
2 |
98.50 |
|
Globodera spp. |
1 |
99 |
||
Table 8. The performance metrics of the classification of cyst nematode of Heterodera spp. and Globodera spp. using models developed based on image textures
|
Algorithm |
True class |
TPR |
FPR |
ROC Area |
PRC Area |
Precision |
F-Measure |
MCC |
Kappa statistic |
|
WiSARD |
Heterodera spp. |
0.980 |
0.010 |
0.992 |
0.989 |
0.980 |
0.980 |
0.970 |
0.9700 |
|
Globodera spp. |
0.990 |
0.020 |
0.994 |
0.996 |
0.990 |
0.990 |
0.970 |
||
|
Multilayer Perceptron |
Heterodera spp. |
0.980 |
0.000 |
0.995 |
0.995 |
1.000 |
0.990 |
0.985 |
0.9849 |
|
Globodera spp. |
1.000 |
0.020 |
0.995 |
0.995 |
0.990 |
0.995 |
0.985 |
||
|
RBF Network |
Heterodera spp. |
0.980 |
0.010 |
0.992 |
0.989 |
0.980 |
0.980 |
0.970 |
0.9700 |
|
Globodera spp. |
0.990 |
0.020 |
0.994 |
0.996 |
0.990 |
0.990 |
0.970 |
Cyst nematode species of G. pallida and G. rostochiensis were distinguished with an average accuracy of up to 85.5% for a model built using WiSARD. Cyst nematodes of G. pallida were correctly classified in 87% and G. rostochiensis in 84% (Table 9). In the case of WiSARD algorithm, model was also characterized by the highest Kappa statistic of 0.7100, TPR of 0.870 and 0.840, Precision of 0.845 and 0.866, F-Measure of 0.857 and 0.853, MCC of 0.710 and 0.710, and the lowest FPR of 0.160 and 0.130 for G. pallida and G. rostochiensis, respectively (Table 10).
Table 9. The distinguishing cyst nematode species of G. pallida and G.rostochiensis belonging to the genus Globodera.
|
Algorithm |
True class |
Predicted class (%) |
Average accuracy (%) |
|
|
G. pallida |
G. rostochiensis |
|||
|
WiSARD |
G. pallida |
87 |
13 |
85.5 |
|
G. rostochiensis |
16 |
84 |
||
|
Multilayer Perceptron |
G. pallida |
83 |
17 |
81.5 |
|
G. rostochiensis |
20 |
80 |
||
|
RBF Network |
G. pallida |
82 |
18 |
76.5 |
|
G. rostochiensis |
29 |
71 |
||
Table 10. The performance metrics of distinguishing cyst nematode of G. pallida and G.rostochiensis based on models built using image texture parameters
|
Algorithm |
True class |
TPR |
FPR |
ROC Area |
PRC Area |
Precision
|
F-Measure |
MCC |
Kappa statistic |
|
WiSARD |
G. pallida |
0.870 |
0.160 |
0.807 |
0.800 |
0.845 |
0.857 |
0.710 |
0.7100 |
|
G. rostochiensis |
0.840 |
0.130 |
0.734 |
0.725 |
0.866 |
0.853 |
0.710 |
||
|
Multilayer Perceptron |
G. pallida |
0.830 |
0.200 |
0.869 |
0.839 |
0.806 |
0.818 |
0.630 |
0.6300 |
|
G. rostochiensis |
0.800 |
0.170 |
0.869 |
0.870 |
0.825 |
0.812 |
0.630 |
||
|
RBF Network |
G. pallida |
0.820 |
0.290 |
0.864 |
0.826 |
0.739 |
0.777 |
0.533 |
0.5300 |
|
G. rostochiensis |
0.710 |
0.180 |
0.864 |
0.856 |
0.798 |
0.751 |
0.533 |
More proof-reads are needed to increase readability.
Response: The whole manuscript has been checked and language has been corrected.
Author Response File: Author Response.pdf
Reviewer 4 Report
1) The description of the models in 2.2.2 may look better in Table format.
2) Please show a few images of the data used.
3) Since the Image processing and feature selection is the main contribution:
a) Can you show a visualization of the outputs of the preprocessing steps
b) It would be nice to compare the performance with an end-to-end system in which the input to the models are the images without any image processing.
Regarding English, it is preferable to use "active voice" in scientific writing; for example: "Models were built using Narrow Neural Network ..." raises the question, Who built them? Are the models taken from elsewhere or built by the authors?
It is clearer to say, "We built several models, including Narrow Neural ..."
2.2.1 says: "The color calibration of the device was performed. Imaging was carried out on a black background at a resolution of 1200 dpi."
It may sound better as: "We performed color calibration and collected the images on a black ..."
As a recommendation, review the whole manuscript and rewrite it in active voice if possible.
Author Response
Reviewer 4
1) The description of the models in 2.2.2 may look better in Table format.
Response: The neural network models with the specified parameters developed using MATLAB and WEKA are presented in Table 2.
Table 2. The parameters of classifiers used to build models using MATLAB and WEKA
|
Application |
Classifier type |
Parameters |
|
MATLAB |
Narrow Neural Network |
first layer size: 10; number of fully connected layers: 1; activation: ReLU; iteration limit: 1000; standardize data: yes; regularization strength (Lambda): 0 |
|
Medium Neural Network |
first layer size: 25; number of fully connected layers: 1; activation: ReLU; iteration limit: 1000; standardize data: yes; regularization strength (Lambda): 0 |
|
|
Wide Neural Network |
first layer size: 100; number of fully connected layers: 1; activation: ReLU; iteration limit: 1000; standardize data: yes; regularization strength (Lambda): 0 |
|
|
Bilayered Neural Network |
first layer size: 10; second layer size: 10; number of fully connected layers: 2; activation: ReLU; iteration limit: 1000; standardize data: yes; regularization strength (Lambda): 0 |
|
|
Trilayered Neural Network |
first layer size: 10; second layer size: 10; third layer size: 10; number of fully connected layers: 3; activation: ReLU; iteration limit: 1000; standardize data: yes; regularization strength (Lambda): 0 |
|
|
WEKA |
WiSARD |
batchSize: 100; bitNo: 8; bleachConfidence: 0.01; bleachFlag: False; bleachStep: 1.0; debug: False; doNotCheckCapabilities: False; mapType: RANDOM; seed: -1; ticNo: 256 |
|
Multilayer Perceptron |
batchSize: 100; debug: False; doNotCheckCapabilities: False; decay: hiddenLayers: a; learningRate: 0.3; momenetum: 0.2; nominalToBinaryFilter:True; normalizeAttributes: True; normalizeNumericClass: True; reset: True; resume: False; seed: 0; trainingTime: 500; validationTreshold: 20 |
|
|
RBF (Radial basis function) Network |
batchSize: 100; clusteringSeed: 1; debug: False; doNotCheckCapabilities: False; maxIts: -1; minStdDev: 0.1; numClusters: 2; ridge: 1.0E-8 |
2) Please show a few images of the data used.
Response: The manuscript has been supplemented. Sample images of the cyst nematodes are presented in Figure 2.
Figure 2. The sample images of cyst nematodes acquired using a digital scanner.
3) Since the Image processing and feature selection is the main contribution:
- a) Can you show a visualization of the outputs of the preprocessing steps
Response: The preprocessing steps included the change of the file format of cyst images to BMP, image segmentation, and ROI determination before image texture parameters computation. The feature selection improved classification accuracy and shortened analysis time.
It has been specified as follows:
“For a dataset including images of three hundred cysts belonging to three nematode species, we performed the image segmentation, ROI determination, and image texture computation separately for each image including one cyst nematode. Before image processing, we changed the file format of cyst images to BMP and processed the images using the MaZda software”
The visualization of the outputs of the preprocessing steps is included in Figure 1.
- b) It would be nice to compare the performance with an end-to-end system in which the input to the models are the images without any image processing.
Response: Thank you for this suggestion. We performed the classification for a dataset without attribute selection. The performance metrics were lower. We obtained the results presented in the paper using the most effective procedure. We used the MaZda software which require image preprocessing to extract the image texture parameters. The change of the file format of cyst images to BMP, image segmentation, and ROI determination were necessary to compute cyst nematode features.
The additional classification models have been developed to distinguish Heterodera species from Globodera species. Then Globodera pallida were distinguished from Globodera rostochiensis.
“In the nest step of the analysis, algorithms providing the highest accuracies of distinguishing all three cyst nematode species were used build further classification models. Firstly, cyst nematodes belonging to Heterodera and Globodera genera were distinguished (Tables 7-8). Then, models were developed to classify species of G. pallida and G. rostochiensis belonging to the genus Globodera (Tables 9-10). Cyst nematodes of Heterodera spp. and Globodera spp. were correctly classified in 99.00% for the model built using Multilayer Perceptron and 98.50% for WiSARD and RBF Network (Table 7). In the case of a model built using Multilayer Perceptron, the highest Kappa statistic of 0.9849 was found. The values reaching 1.000 were observed in the case of TPR for Globodera spp. and Precision for Heterodera spp. The lowest FPR of 0.000 was determined for Heterodera spp. (Table 8).
Table 7. The results of the classification of cyst nematodes of Heterodera and Globodera spp. based on models built using image texture parameters
|
Algorithm |
True class |
Predicted class (%) |
Average accuracy (%) |
|
|
Heterodera spp. |
Globodera spp. |
|||
|
WiSARD |
Heterodera spp. |
98 |
2 |
98.50 |
|
Globodera spp. |
1 |
99 |
||
|
Multilayer Perceptron |
Heterodera spp. |
98 |
2 |
99.00 |
|
Globodera spp. |
0 |
100 |
||
|
RBF Network |
Heterodera spp. |
98 |
2 |
98.50 |
|
Globodera spp. |
1 |
99 |
||
Table 8. The performance metrics of the classification of cyst nematode of Heterodera spp. and Globodera spp. using models developed based on image textures
|
Algorithm |
True class |
TPR |
FPR |
ROC Area |
PRC Area |
Precision |
F-Measure |
MCC |
Kappa statistic |
|
WiSARD |
Heterodera spp. |
0.980 |
0.010 |
0.992 |
0.989 |
0.980 |
0.980 |
0.970 |
0.9700 |
|
Globodera spp. |
0.990 |
0.020 |
0.994 |
0.996 |
0.990 |
0.990 |
0.970 |
||
|
Multilayer Perceptron |
Heterodera spp. |
0.980 |
0.000 |
0.995 |
0.995 |
1.000 |
0.990 |
0.985 |
0.9849 |
|
Globodera spp. |
1.000 |
0.020 |
0.995 |
0.995 |
0.990 |
0.995 |
0.985 |
||
|
RBF Network |
Heterodera spp. |
0.980 |
0.010 |
0.992 |
0.989 |
0.980 |
0.980 |
0.970 |
0.9700 |
|
Globodera spp. |
0.990 |
0.020 |
0.994 |
0.996 |
0.990 |
0.990 |
0.970 |
Cyst nematode species of G. pallida and G. rostochiensis were distinguished with an average accuracy of up to 85.5% for a model built using WiSARD. Cyst nematodes of G. pallida were correctly classified in 87% and G. rostochiensis in 84% (Table 9). In the case of WiSARD algorithm, model was also characterized by the highest Kappa statistic of 0.7100, TPR of 0.870 and 0.840, Precision of 0.845 and 0.866, F-Measure of 0.857 and 0.853, MCC of 0.710 and 0.710, and the lowest FPR of 0.160 and 0.130 for G. pallida and G. rostochiensis, respectively (Table 10).
Table 9. The distinguishing cyst nematode species of G. pallida and G. rostochiensis belonging to the genus Globodera.
|
Algorithm |
True class |
Predicted class (%) |
Average accuracy (%) |
|
|
G. pallida |
G. rostochiensis |
|||
|
WiSARD |
G. pallida |
87 |
13 |
85.5 |
|
G. rostochiensis |
16 |
84 |
||
|
Multilayer Perceptron |
G. pallida |
83 |
17 |
81.5 |
|
G. rostochiensis |
20 |
80 |
||
|
RBF Network |
G. pallida |
82 |
18 |
76.5 |
|
G. rostochiensis |
29 |
71 |
||
Table 10. The performance metrics of distinguishing cyst nematode of G. pallida and G. rostochiensis based on models built using image texture parameters
|
Algorithm |
True class |
TPR |
FPR |
ROC Area |
PRC Area |
Precision
|
F-Measure |
MCC |
Kappa statistic |
|
WiSARD |
G. pallida |
0.870 |
0.160 |
0.807 |
0.800 |
0.845 |
0.857 |
0.710 |
0.7100 |
|
G. rostochiensis |
0.840 |
0.130 |
0.734 |
0.725 |
0.866 |
0.853 |
0.710 |
||
|
Multilayer Perceptron |
G. pallida |
0.830 |
0.200 |
0.869 |
0.839 |
0.806 |
0.818 |
0.630 |
0.6300 |
|
G. rostochiensis |
0.800 |
0.170 |
0.869 |
0.870 |
0.825 |
0.812 |
0.630 |
||
|
RBF Network |
G. pallida |
0.820 |
0.290 |
0.864 |
0.826 |
0.739 |
0.777 |
0.533 |
0.5300 |
|
G. rostochiensis |
0.710 |
0.180 |
0.864 |
0.856 |
0.798 |
0.751 |
0.533 |
Regarding English, it is preferable to use "active voice" in scientific writing; for example: "Models were built using Narrow Neural Network ..." raises the question, Who built them? Are the models taken from elsewhere or built by the authors?
It is clearer to say, "We built several models, including Narrow Neural ..."
Response: It has been corrected in the whole manuscript.
2.2.1 says: "The color calibration of the device was performed. Imaging was carried out on a black background at a resolution of 1200 dpi."
It may sound better as: "We performed color calibration and collected the images on a black ..."
Response: It has been corrected.
As a recommendation, review the whole manuscript and rewrite it in active voice if possible.
Response: It has been corrected in the whole manuscript.
Author Response File: Author Response.pdf
Round 2
Reviewer 2 Report
The authors have addressed all my concerns. The paper has improved a lot and I think now it is ready for publication.
Reviewer 4 Report
The authors made the suggested changes. The manuscript looks better now.
