The Development of a Defect Detection Model from the High-Resolution Images of a Sugarcane Plantation Using an Unmanned Aerial Vehicle

This article presents a defect detection model of sugarcane plantation images. The objective is to assess the defect areas occurring in the sugarcane plantation before the harvesting seasons. The defect areas in the sugarcane are usually caused by storms and weeds. This defect detection algorithm uses high-resolution sugarcane plantations and image processing techniques. The algorithm for defect detection consists of four processes: (1) data collection, (2) image preprocessing, (3) defect detection model creation, and (4) application program creation. For feature extraction, the researchers used image segmentation and convolution filtering by 13 masks together with mean and standard deviation. The feature extraction methods generated 26 features. The K-nearest neighbors algorithm was selected to develop a model for the classification of the sugarcane areas. The color selection method was also chosen to detect defect areas. The results show that the model can recognize and classify the characteristics of the objects in sugarcane plantation images with an accuracy of 96.75%. After the comparison with the expert surveyor’s assessment, the accurate relevance obtained was 92.95%. Therefore, the proposed model can be used as a tool to calculate the percentage of defect areas and solve the problem of evaluating errors of yields in the future.


Introduction
Sugarcane is an economic crop of Thailand and the raw material of the sugar factory. Thailand has suitable geographical and climatic conditions for sugarcane cultivation. Moreover, it is the 4th largest sugar producer in the world after Brazil, India, and the European Union [1]. Currently, the Thai government supports the agriculture and biotechnology industry to improve the economy and increase the competitiveness of the country. The industry is operated by applying modern agricultural technology such as the use of sensor systems and advanced data analysis techniques [2]. The average age of sugarcane cultivation is 12 months per harvest. The assessment and survey of sugarcane plantation by the expert surveyor is operated in 2 stages: the first 4 months after planting, and the last 2 months before harvesting. For the harvesting methods, the sugarcane will be harvested by human labor and a sugar harvester [3].
Kamphaeng Phet province is one of the most widely cultivated sugarcane provinces in Thailand. It is located in the lower northern region of Thailand, with sandy soil suitable for farming and growing crops with an area of 2.1 million acres. The land use is divided into three groups: 1.24 million acres of agriculture, 0.47 million acres of forest, and 0.39 million acres of residential land. Kamphaeng Phet's gross domestic product of 3547$ million is ranked 2nd in the northern region and ranked 4th in the country. The climatic condition in Kamphaeng Phet is categorized as tropical grassland

Material and Method
An unmanned aerial vehicle of the DJI company (Phantom4 model) was proposed for the collection of datasets to create a defect detection model. The DJI Phantom4 is capable of flying up to 30 min, with a wind resistance of 10 m/s and equipped with an ultra-high-definition quality camera which is 20,962 cm 2 calculated from 72 dpi. In this research, two software were proposed to develop the model, including (1) Google Earth Engine editor for creating coordinates of the sugarcane plantation, and (2) MATLAB 2015b for model development and image processing. The conceptual framework of the study is shown in Figure 1. In Figure 1, the defect detection framework for sugarcane plantation consists of 4 main steps: data collection, image-preprocessing, defect detection model creation, and application program creation. The details of each steps are discussed below.

Data Collection
The sugarcane datasets were collected from a sugar factory in Kamphaeng Phet Province. After the data cleaning process, 2724 plots, out of the total 3442 plots, were left. The collection of data by unmanned aerial vehicles was done by random sampling considering different environmental factors, such as sugarcane varieties, sugarcane ratoon, soil series, and yield levels ( Table 1). The selection of patterns from the sample plantation was carried out by the Cartesian product method [17] from the factors in Table 1. The resulting Cartesian product had 594 patterns (3 × 3 × 3 × 22 = 594), in which only 90 data matched the pattern from the actual dataset. The details of the population (red spots) are shown in Figure 2A, and the samples (red spots) are shown in Figure 2B.  Figure 2B shows a sampling of the selected sugarcane plots in the province of Kamphaeng Phet. The sampling images were used to develop a model for analyzing the defect present in a sugarcane plantation. The sample datasets are shown in Figure 3. A survey was done in September to October 2018 using the UAV, which consists of the four conditions: height from the ground at 200 m to 300 m, undefined environment, shooting time, and the size of the images (3078 × 5472 pixels). The images collected from the survey were used as the datasets for the study and development of the defect detection model.

Image Preprocessing
The image-preprocessing process is one of the crucial steps for the preparation of datasets before using image processing techniques. The images collected from the UAV have a shadow region formed during shooting at different times. The shadows were mostly created by trees and different sugarcane heights. Therefore, it is necessary to have shadow detection and shadow adjustment methods before the image is being processed. The sample image with the shadow region is shown in Figure 4. The sugarcane area was divided into 2 categories: hard shadow and soft shadow [18]. The image was taken from a height of 200 m for both hard and soft shadow ( Figure 4). The original image was converted to a grayscale and Otsu's thresholding method was applied to detect the shadow region [19]. However, the sugarcane under the shadow region could not be analyzed as defects because of thresholding. Therefore, the shadow detection and the shadow adjustment were proposed before applying Otsu's thresholding method.

Shadow Detection
Shadow detection is one of the methods in the image preprocessing step before applying the image processing technique. The shadow region affects the analysis of the defect areas in the image. Some researchers have suggested shadow detection of outdoor images [20] and infrared images [21]. In this study, a shadow detection algorithm using LAB [22] color space and statistical values [23] was proposed as shown in Figure 5. The shadow detection method consists of the following steps.

•
Step 1: Convert the color image to LAB color space.

•
Step 2: Calculate the mean of each color plane followed by the standard deviation of L plane.

•
Step 3: Detect the shadow pixel by thresholding. The conditions for thresholding are shown in the Figure 5 and the algorithm is shown as a pseudo-code in Algorithm 1.

•
Step 4: Divide the areas into 2 regions: the shadow regions g i (x, y) and the non-shadow region regions h i (x, y). The shadow region has a pixel value of 0 (black) and the non-shadow region has a pixel value of 255 (white). The results from the shadow detection method are used as an input for shadow adjustment process.

Shadow Adjustment
The shadow adjustment process was applied after the shadow detection process. Moreover, this process is considered as a crucial step for image preparation before entering the sugarcane image classification. Currently, there are a number of methods already developed for shadow adjustment methods, such as the shadow adjustment from RGB-D color images [24], the shadow adjustment of the high buildings from satellite images [25], and shadow adjustment for the survey of utilization areas [26]. The adjustment of a shadow depends on the amount of lights in shadows and non-shadow regions [27]. The flowchart of shadow adjustment method is shown in Figure 6.
The shadow adjustment process consists of 3 steps for calculating the light source of the shadow region g i (x, y) and the non-shadow region h i (x, y) areas. The constant ratio is calculated from the brightness values of the pixel in the shadow areas.

•
Step 1: Calculate the light source of the shadow and non-shadow areas in the output image given by the shadow detection method. The light source value was calculated from Equations (1) and (2).
where e 1_i is the xy-summation of the shadow region.
MN is the number of pixels on the shadow region. k_i is the scale factor defined by R channel (KR), G channel (KG), B channel (KB). p is the weight of each gray value in the light source.
where e 2_i is the xy-summation of the non-shadow region.
MN is the number of pixels on the non-shadow region. k_i is the scale factor defined by R channel (KR), G channel (KG), B channel (KB). p is the weight of each gray value in the light source.

•
Step 2: Determine the value of p that is suitable for adjusting the intensity of the shadow areas. Brightness, contrast, and average gradient of the shadow area were compared with the non-shadow areas. The appropriate p-values generated from the experiments is shown in Section 3.2.

•
Step 3: Calculate the constant ratio of the light source values in both areas and proceed to adjust the intensity of the shadow area as in Equation (3).
where g_h i is the new shadow region. e 2_i /e 1_i is the rate constant of shadow and non-shadow regions. g i (x, y) is a shadow region. After the shadow adjustment process, the shadow areas are usually brighter and sharper than original images. Therefore, those images were used for the development of the model for detecting the defects in the sugarcane plantation.

Defect Detection Model Creation
The development of the defect detection model in sugarcane plantation consists of 4 steps: (1) feature extraction, (2) classification of sugarcane areas, (3) defect area classification, and (4) data integration. The details are as follows.

Feature Extraction
The feature extraction process is one of the essential steps for classification of objects within sugarcane plot such as sugarcane area, weeds, soil, water sources, roads, and trees. In this research, 90 UAV images were selected for texture analysis and image segmentation. For the image segmentation process, the image was resized to 3000 × 5000 pixels for non-border and non-missing parts of the sub-image. The researchers divided the image into 50 × 50 grid cells because the researcher can carefully label the images which are to be categorized. From 50 × 50 grid cells, 60,000 sub-images were generated from one image. However, the researchers selected a region of interest (50 × 50) from the image as the datasets as shown in Table 2, for training the model. The datasets were divided into two categories as shown in Table 2: Dataset 1 consists of sugarcane, trees, and weeds images whereas dataset 2 contains weeds and trees images. In both of the datasets, 80-20 ratio was used to split the datasets into training and testing sets. Each category of datasets has 5 varying sizes such as (1) 10 × 10, (2) 20 × 20, (3) 25 × 25, (4) 40 × 40, and (5) 50 ×50, respectively, as shown in Table 4. This size is calculated using the greatest common factor (GCF) algorithm. The conceptual framework of feature extraction is presented in Figure 7. The conceptual framework is explained by using 50 × 50 pixel which consists of 4 steps:

•
Step 1: Convert RGB color image to grayscale format.

•
Step 2: Convolute the grayscale images by 13 filtering masks to get the features. In this research, standardized filters [28] (3 × 3) as shown in Figure 8 was used.

•
Step 3: The statistical values such as mean and standard deviation are calculated from step 2. This process achieves 26 characteristics.

•
Step 4: Select the features that are related to the class of the category using the WEKA program [29]. WEKA program uses the "Information Gain-Based Feature" selection method to find the significant features which is known as entropy. The features must pass the threshold value of more than 0.05. This value is selected using a T-score distribution method which separates the group of data obtained from entropy and eliminates those groups that have least relationship with the class (sugarcane, trees, and weeds). When the threshold value 0.05 was used as a condition, it gives the most significant features for the classification of sugarcane areas.

The Sugarcane Areas Classification Process
The sugarcane area classification is a process used to classify sub-areas (sugarcane, trees, and weed) within an image. The dataset used in the development of the process is shown in Table 2; which consists of training and validation sets for training purposes and the testing set to evaluate the model. In the experiments, the K-nearest neighbors method [30] was selected to develop a model for classification of sugarcane areas.

The Defect Areas Classification Process
This process was carried out after identification of sugarcane areas from the previous step. In this process, color analysis and color selection methods were used to eliminate other colors except green shades. The color selection method is performed by converting the RGB color image into the HSV color space and then selecting Hue channel into 3 ranges (yellow-green: 60-80 degrees, green: 81-140 degrees, and green-cyan: 141-169 degrees) of green shades [31]. The color selection process provides only the areas of trees, weeds, and green water sources. However, the stunted weeds, ground, and roads were eliminated since their color space does not fall under the given range.

Data Integration Process
The data integration is a process that combines the results of the sugarcane areas classification and defect areas classification. This process is performed by the logical AND operator to integrate the defects and sugarcane areas.

Application Program Creation
The graphical user interface (GUI) was developed to assist the selection of sugarcane areas from the images. Moreover, the user interface can be used to draw the polygonal lines [32] to select the region of interest and help to reduce errors by avoiding other objects in the image. After drawing a polygon into the desired area, the program will analyze the selected area and identify the faulty spots such as water sources, weeds, defect sugarcane, etc. The defect detection rate in the GUI is shown as a percentage in the top right corner of the program window as shown in Figure 17.

Experiment Results
The aim of developing a defect detection model in sugarcane plantation images is to reduce the yield estimation error. The model uses high-resolution images captured from the UAV. The proposed methods consists of 4 steps: data collection, image-preprocessing, defect detection model creation, and application program creation. For image preprocessing, the researchers designed two experiments: (1) performance testing for shadow detection and (2) the performance testing for shadow adjustment. Whereas, for the evaluation of defect detection models, the performance testing for sugarcane area classification and defect area classification experiments were designed.

The Performance Testing for Shadow Detection
The shadow detection is one of the image preprocessing steps. The experiment was carried out with experts by manually plotting shadow and non-shadow areas as shown in Figure 9A. The sugarcane plot contains 100 by 100 grid cells. The conditions for identification of the shadow region in the grid cell for both expert and proposed model must have 50% of shadow in the square (thick square) [33]. The example of experiments for shadow detection is shown in Figure 9. The experiment was conducted to compare the results of all 90 images. The experimental results found that the average precision was 75.92% with a recall of 92.23% and F1-measure of 80.11%. The overall accuracy obtained from the experiments was 98.44%. The relationship between the sensitivity (true positive rate: TP) and 1-specificity rate (false positive rate: FP) is shown in Figure 10. The ROC curve of the experimental result shows that the average sensitivity and 1-specificity was 85.80% and 99.88% respectively. The result has a high performance due to the adjacent left corner and the average areas under the curve (AUC) which was 99.67%. Therefore, the method proposed for shadow detection areas is highly effective in detecting the shadow region in the sugarcane areas.

The Performance Testing for Shadow Adjustment
The results obtained from shadow detection method is passed as an input to this method. This process adjusts the intensity of the shadow areas from approaching the non-shadow areas. In this method, the parameter p of the Minkowski norm [34] (Equations (2) and (3)) is adjusted from 1 to 10 to get the proper p-value. The p-value is selected based on 3 factors such as brightness, contrast, and the average gradient of each image. The sample p-value for one of the sugarcane plant images is shown in Table 3. Table 3. The example experiment of the shadow adjustment process. This process selects the result that is closest to the value of the non-shadow areas from all three factors in each color plane. The experimental results are shown in Figure 11. The experiment found that most of the images have an exponential value of p = 1, but there are some images with p-value of 2, 3, and 4, due to brighter non-shadow region than the shadow region. As a result, it is necessary to adjust the p-value to be closest to the non-shadow area.

The Performance Testing for Sugarcane Areas Classification
The sugarcane areas classification is a process to identify the categories of objects present in the images such as sugarcane, trees, weeds, and others. The experiment is divided into 2 parts: (1) the experiment for feature selection and (2) the experiment to develop model for sugarcane area classification. The experiment was conducted with 2 datasets as shown in Table 2.

The Experiment of Feature Selection
Feature selection is a process of gathering important attributes from the image which can help to distinguish from the other image. The selection of features is carried out by correlation with classes of each category from the sub-images of the 2 datasets. The features that passed the threshold value of 0.05 were used to train the model and eliminate the less important features. The experimental results are shown in Table 4. In Table 4, the features that do not satisfy the given conditions are underlined and the other features were used in training the model.

The Experiment of Sugarcane Area Classification Model Creation
For sugarcane classification, 2 datasets were used to train the model using the K-nearest neighbor algorithm. After training the model, the model which gives the best accuracy to classify the sugarcane area was selected for the research. The model was trained and validated using the 10-fold cross-validation method. The experimental results are shown in Table 5. The experiment shows that dataset 1 with size 50 × 50 pixels obtained the highest accuracy of 91.38% with the 10-fold cross-validation whereas, the testing set gave of 85.65% which results in overfitting. Dataset 2 with a size 50 × 50 pixels had the highest accuracy of 96.75% with a 10-fold cross-whereas, the testing set gave 95.01%. Therefore, the dataset 2 was proposed for the development of sugarcane area classification model but both of the datasets could not identify trees. The results given by the experiments were compared with what the sugarcane surveyor experts marked. The only condition for identification of sugarcane area by the experts and models that there must be more than 50% of sugarcane area in the grid cell. Figure 12 shows the sugarcane classification done by the model and the experts. In Figure 12, the non-sugarcane areas are identified by thick squares. In the performance testing of model 1 (created from dataset 1): the expert identified the trees, weeds, and other areas that are non-sugarcane for comparison with the result of the model. However, in model 2 (created by dataset 2): the expert identified the weeds and other areas that are non-sugarcane for comparison with the result of model. The results obtained from classification of sugarcane areas by experts and the proposed model are shown in Table 6. The relationship between sensitivity (true positive rate: TP) and 1-specificity (false positive rate: FP) calculated from confusion metrics is shown in Figure 13.
The ROC curve of experiment result found that model 2 is more efficient than model 1. The average sensitivity in the model 2 was 74.03%, and the 1-specificity was 87.54%. The curve of model 2 was adjacent to the left corner and the average areas under curve (AUC) was 92.27% (see Figure 13B). The average sensitivity in the model 1 69.38%, and the 1-specificity was 71.01%. The average areas under curve (AUC) was 89.42% (see Figure 13A). Both models cannot identify the trees in sugarcane plantation images. Therefore, model 2 is chosen to create a model for sugarcane area classification.

The Performance Testing for Defect Areas Classification
The sugarcane defect area classification is a process applied after the shadow detection, the shadow adjustment, and the sugarcane area classification process. This experiment consists of results from the color selection and the data integration. The experiment was compared with what the sugarcane surveyor experts marked to identify the defect areas that were divided into 100x100 grid cells in the original images. The only condition for identification of the defect area by experts and the models is that there must be more than 50% of defects (thick square) in the square. See an example of the operation in Figure 14. The experiment was conducted to compare the results of all 90 images. The experiment results found that the average precision was 76.06% with recall value of 81.49% and F1-measure 79.73%. The overall accuracy of defect areas was 87.20%. The relationship between sensitivity (true positive rate: TP) and 1-specificity (false positive rate: FP) is shown in Figure 15.
This ROC curve of experiment result found that the average sensitivity and 1-specificity was 87.20% and 75.78% respectively. The result has a high performance due to the adjacent left corner and the average areas under curve (AUC) was 92.95%. Therefore, the parameters and the model can be applied for application program creation.

Result and Discussion
The defect detection model was developed by image processing techniques with high-resolution images taken from an unmanned aerial vehicle. The model consists of five main processes such as shadow detection, shadow adjustment, feature extraction, sugarcane area classification, and defect area classification process. The results obtained from the experiments were used to develop the application to assess the defect on the sugarcane plantation. The results of the defect detection model is shown in Figure 16. The image preprocessing techniques consists of the shadow area detection and shadow adjustment process. The experiment found that the shadow detection method can detect the shadow cast by trees and high sugarcane with the accuracy of 99.67% as shown in Figure 16B. The shadow adjustment process applies intensity adjustment in the shadow areas as a result shown in Figure 16C.
In the sugarcane area classification, 50 × 50 pixels sub-images were used for extraction of features for the development of model. After training the features, dataset 2 (having sugarcane and weeds) gave the highest accuracy in the development of the model. However, the sugarcane area classification model could not identify the trees in either of the datasets, but it could identify the sugarcane area and weeds with the accuracy 92.27% for ROC curve in Figure 13B. The result of the sugarcane area classification is shown in Figure 16D.
The defect detection, the color selection process (see in Figure 16E) and data integration process were proposed. The data integration process was operated by combining the results of sugarcane area classification and color selection process (see in Figure 16F). The experiment found that the defect detection process was able to identify the defect in sugarcane plantation image with the accuracy 92.95%.
The overall experiment of the proposed method can detect the defects such as water sources, defect areas, ground, residences, and road in the sugarcane plantation. However, the method could not identify trees in sugarcane plantation. Therefore, the researcher developed the graphical user interface (GUI) to solve the problem of identifying trees in the sugarcane plantation while drawing the polygon on the interested areas. The GUI is shown in Figure 17. The GUI is used for drawing the interesting regions for classifying the sugarcane areas whereby the trees in the image are not selected. The example plantation (10008002) as given in Figure 17, which shows the percentage of the defects in interested areas. With the introduction of the GUI, the experts are satisfied with the program with a satisfaction level of good (4.53).The performance comparisons of the proposed methods with the methods discussed in the literature review is presented in Table 7. The description presented on for table is reflected as per the information given in the reference paper.
The proposed method is different from the methods discussed in the literature. The researchers in [10,12,13] have analyzed the sugarcane plants in the mature stage, the same as the proposed method. However, different features and classifiers were used resulting in different accuracy. When comparing our results with the other methods, the proposed classifier generated an overall accuracy of 96.75% which is higher than the other methods that use the plants in the mature stage. The higher accuracy is due to the introduction of shadow detection and shadow adjustment preprocessing steps. In our approach, the researchers have extracted features using standard deviation, mean value, and 13 filtering masks. The researchers have not used other statistical values such as kurtosis, skewness, variance, etc. because it generated 82% [16] accuracy which is less than the proposed methods. The other methods were not compared since the detection is done during the tilling stage.

Conclusions
For the development of the model to analyze the defect areas of the sugarcane plantations, high-resolution image taken from an unmanned aerial vehicle was used. This model uses shadow detection, shadow adjustment, feature extraction, and classification methods. For feature extraction, the algorithm was developed by using 13 filter masks together with convolution operations into the sub-images. In each mask, the mean and standard deviation were calculated which makes up to 26 characteristics. The result of the process obtained 26 characteristics from the mean and standard deviation of the data to represent the value of the sub-image. The feature selection method was used based on the calculation of entropy between attributes and the class labeling. The 22 features extracted from the feature selection methods were trained and tested using the K-nearest neighbors classification algorithm in WEKA. The algorithm provided an accuracy of 96.75%. The model can classify the objects in the image, including water sources, defect areas, ground, residences, and roads precisely, but it cannot classify trees. Some parts of the trees and sugarcane have similar characteristics when extracted, thus making it difficult to distinguish between the sugarcane and trees. However, this method can be applied in conjunction with the selection of areas, excluding trees from the image. Therefore, while selecting the sugarcane plot on the GUI application, the user must avoid trees in order to have a high accuracy of sugarcane detection areas. The developed model can be used as a tool to calculate the percentage of defect areas, which can possibly solve the problem on sugarcane yield deviation. For future work, an automatic method such as Fourier analysis can be applied for detecting the tree automatically and the model will be improved to classify the objects with greater accuracy. Moreover, the industrial plants will benefit in resource management such as data collection, cost of climate factors survey, and yield deviation reduction.
Funding: This research received no external funding.