Review Reports - Evaluation of Convolutional Neural Networks’ Hyperparameters with Transfer Learning to Determine Sorting of Ripe Medjool Dates

Round 1

Reviewer 1 Report

Agriculture (ISSN 2077-0472)

Manuscript Number: agriculture-1078735

Title: Evaluation of Convolutional Neural Networks Hyperparameters with Transfer Learning to Determine Sorting of Ripe Medjool Dates

Article Type: Review Article

The subject of research includes in this journal. Research work is interesting. In the paper titled “Evaluation of Convolutional Neural Networks Hyperparameters with Transfer Learning to Determine Sorting of Ripe Medjool Dates. The authors used Convolutional Neural Networks to assess quality and classify products. This method is very often already used. Therefore, there is nothing innovative in the presented solution. The paper needs revisions and need to take care of following:

The quality of the language is insufficient. Have a native speaker or similar assist you.
The Abstract is quite random with broad and varying statements and not very well focused.
Why did the authors use the JPG file format. This format is not a good choice because the image loses quality.
Please explain what was at the output of the network.
The article should be rewritten into the scientific work structure: Introduction, materials and methods; results; discussion conclusions.

Author Response

Review 1 comments:

The subject of research includes in this journal. Research work is interesting. In the paper titled “Evaluation of Convolutional Neural Networks Hyperparameters with Transfer Learning to Determine Sorting of Ripe Medjool Dates. The authors used Convolutional Neural Networks to assess quality and classify products. This method is very often already used. Therefore, there is nothing innovative in the presented solution. The paper needs revisions and need to take care of following:

The quality of the language is insufficient. Have a native speaker or similar assist you.

Author response:

Corrections were made, with the help of an external expert.

The Abstract is quite random with broad and varying statements and not very well focused.

Author response:

To address the reviewer's commentary on the abstract. We modified abstract as follows:

Convolutional neural networks (CNN) have proven their efficiency in various applications in agriculture. In crops such as date, they have been mainly used in the identification and sorting of ripe fruits. The aim of this study was the performance evaluation of eight different CNN, considering transfer learning for their training, as well as five hyperparameters. The CNN architectures evaluated were VGG-16, VGG-19, ResNet-50, ResNet-101, ResNet-152, AlexNet, Inception V3, and CNN from scratch. Likewise, the hyperparameters analyzed were the number of layers, the number of epochs, the batch size, optimizer, and learning rate. The accuracy and processing time were considered to determine the performance of CNN architectures, in the sorting of mature dates cultivar Medjool. The model obtained from VGG-19 architecture with a batch of 128 and Adam optimizer with a learning rate of 0.01 presented the best performance with an accuracy of 99.32%. We conclude that the VGG-19 model can be used to build computer vision systems that help producers improve their sorting process to detect the Tamar stage of a Medjool date.

Why did the authors use the JPG file format. This format is not a good choice because the image loses quality.

Author response:

The reviewer is correct that image quality is an essential practical challenge that is sometimes not considered in computer vision systems. However, we train the network with jpg images because, in reality, the systems are fed with low-quality images. We refer to low-quality images with blur, noise, contrast, or compression. We consider that if you are trained with this type of image, the system will be able to classify the Medjool date in images with these features.

Regarding the reviewer comment, we added the following paragraph in section 2.2 Image dataset, on page 2, lines 85 to 90.

We train the network architectures with JPG images because they are fed with low-quality images in real scenarios. We refer to low-quality images with blur, noise, contrast, or compression. We consider that if you are trained in architecture with this type of image, the system will be able to classify the Medjool date in images with these features. Further, a study shows that convolutional neural networks are minimally affected in their performance by using JPG format [19].

Please explain what was at the output of the network.

Author response:

Suppose your question is about network parameters. The modified network structure is described on page 5, from line 167 to 181, and presented in Table 1 is presented a summary of CNN networks.

All the above networks are too deep to train them from scratch with our dataset. Therefore, we have used transfer learning, which consists of taking the features learned in other contexts and using them in a new and similar problem [26]. Transfer Learning is usually done for tasks where the dataset has too little data to train a full-scale model from scratch. This is our case since we only have 1,002 Medjool date images.

Transfer Learning is commonly used in two ways: (1) Pre-training model, which consists of using a pre-trained model that replace its last layers with others, so that the characteristics of the new dataset and, (2) Convolutional Network Tuning, that is a strategy to tune the weights of the layers using backward propagation.

For this study, the application of transfer learning was the Pre-training model. We have used the pre-trained networks with ImageNet, which is a large visual database designed for use in visual object recognition [24]. We remove the final classification layer, the neuron softmax layer at the end, which corresponds to ImageNet, and in-stead replace it with a new softmax layer for our image dataset. A summary of the utilized CNN architectures is shown in Table 1.

The article should be rewritten into the scientific work structure: Introduction, materials and methods; results; discussion conclusions.

Author response:

It is essential to mention that the review evaluates the article as a review article. However, our work is an original research article.

Our article has the sections recommended by the agriculture template. Introduction (section 1, page 1), Materials and Methods (section 2, page 2), Results (section 3, page 7), Discussion (section 4, page 10), Conclusions (section 5, page 12) and references.

Reviewer 2 Report

The manuscript presents the application of Convolutional Neural Networks to determinate sorting of ripe fruits. Presented results indicate potential of using CNNs modelling for sorting the Medjool date after harvesting. The manuscript is very well written and text is clear and easy to read. New references were used for the manuscript preparation.

In my opinion this manuscript needs some minor correction:

The novelty and the objectives of the work should be emphasized in the introduction section.
More information about CNNs application in food and agriculture should be included into manuscript
Line 71 description of abbreviations when first used in text should be included
Line 87 where the used images 2D or 3D
Line 91 how was background removed from the picture
Line 296. Percentage accuracy of 99.32% was achieved. What caused the error of 0.7%?

Author Response

In my opinion this manuscript needs some minor correction:

The novelty and the objectives of the work should be emphasized in the introduction section.

We added a paragraph in the introduction section to emphasized the contribution and novelty of the article.

On page 2, from line 65 to 73, we add the following paragraph for contribution.

The main contribution of this article was the identification of the hyperparameters that best influenced the training of a CNN architecture that transfers learning to Medjool's mature date sorting. To achieve it, we perform a comparison of the performance of eight CNN architectures. Two versions of the CNN architecture called the Visual Geometric Group (VGG) from Oxford University, VGG-16, and VGG-19. Three versions of the CNN architecture called Residual Network from Microsoft research, ResNet-50, ResNet-101, and ResNet-152. AlexNet, Inception Version 3, and a CNN from scratch. The hyperparameters analyzed were the number of layers, the number of epochs, the batch size, optimizer, and learning rate.

On page 2, from line 60 to 64, we the following paragraph for novelty.

Regarding dates, we identify that some studies use machine learning algorithms and image processing techniques to sort among date palm fruit or to detect among their different maturity stages [15-17]. Further, there research works that propose using CNNs [8-9, 18]. However, these studies do not present models to detect the maturity stage of the Medjool date.

More information about CNNs application in food and agriculture should be included into manuscript

Authors response:

We added a paragraph in the discussion section. In this paragraph, we mention using convolutional networks in different agriculture areas, for example, detection of plant diseases, yield estimation, fruit classification, and level of maturity. These changes are on page 9, from line 275 to 297.

Convolutional Neural Networks (CNN) are used in several agriculture areas such as leaf and plant disease detection, land cover classification, crop type classification, plant recognition, segmentation of root and soil, crop yield estimation, fruit counting, obstacle detection in row crops and grass mowing, and identification of weeds, to mention a few [10-11]. For example, in Mohanty et al. [12], they present the training of CNN architectures AlexNet and Google Net with a PlanVillage image dataset to detect 26 types of diseases in 14 kinds of crops. Their results show an accuracy of 99.35% to identify healthy and diseased plants. Meanwhile, Rahnemonfar and Sheppard [13] propose using the CNN architectures inception and Residual Networks (ResNet) architectures to estimate the yield of a tomato plant using synthetic images. Their results indicate that with 91% accuracy, they can evaluate the yield.

Another example is presented in [14], where authors propose trained several convolutional networks to identify four fruits, mango, orange, apple, and banana. They were classified into two categories: fresh and rotten. The best performing models were inception version 3 and the Visual Geometric Group of 16-layer (VGG-16) architectures, which received the learning transfer. Their results show identification and classification percentages of 90% accuracy. A similar study is presented in [15], where the use of a VGG-16 network to classify vegetables and fruits is proposed. A total of 26 categories were classified: pumpkin, celery, cauliflower, pineapple, pomegranate, grapefruit, banana, cucumber, broccoli, onion, carrot, etc. The authors claim to have 95.6% accuracy in classifying these fruits and vegetables. Regarding dates, we identify that research works that propose using CNNs to sort among dates or to detect among their different maturity stages [8-9, 16].

Line 71 description of abbreviations when first used in text should be included

Author response:

The acronyms used are defined in the document. On page 2, from line 67 to 73. We made the following changes.

To achieve it, we perform a comparison of the performance of eight CNN architectures. Two versions of the CNN architecture called the Visual Geometric Group (VGG) from Oxford University, VGG-16, and VGG-19. Three versions of the CNN architecture called Residual Network from Microsoft research, ResNet-50, ResNet-101, and ResNet-152. AlexNet, Inception Version 3, and a CNN from scratch. The hyperparameters analyzed were the number of layers, the number of epochs, the batch size, optimizer, and learning rate.

Line 87 where the used images 2D or 3D

Author response:

On page 5, from line 176 to 181, we changed a few words to specify where our images were used. The modified text is as follows

Line 91 how was background removed from the picture

Author response:

The background of the images was not removed,

Line 296. Percentage accuracy of 99.32% was achieved. What caused the error of 0.7%?

Author response

We consider that the error we got is acceptable. The error can be caused by images, as did no preprocessing was done with the background and the hyperparameters defining how networks must be trained.

Author Response File: Author Response.docx

Round 2

Reviewer 1 Report

I have no comments.