Plastic Contaminant Detection in Aerial Imagery of Cotton Fields Using Deep Learning

Abstract

Plastic shopping bags are often discarded as litter and can be carried away from roadsides and become tangled on cotton plants in farm fields. This rubbish plastic can end up in the cotton at the gin if not removed before harvest. These bags may not only cause problems in the ginning process but might also become embedded in cotton fibers, reducing the quality and marketable value. Therefore, detecting, locating, and removing the bags before the cotton is harvested is required. Manually detecting and locating these bags in cotton fields is a tedious, time-consuming, and costly process. To solve this, this paper shows the application of YOLOv5 to detect white and brown colored plastic bags tangled at three different heights in cotton plants (bottom, middle, top) using Unmanned Aircraft Systems (UAS)-acquired Red, Green, Blue (RGB) images. It was found that an average white and brown bag could be detected at 92.35% and 77.87% accuracies and a mean average precision (mAP) of 87.68%. Similarly, the trained YOLOv5 model, on average, could detect 94.25% of the top, 49.58% of the middle, and only 5% of the bottom bags. It was also found that both the color of the bags (p < 0.001) and their height on cotton plants (p < 0.0001) had a significant effect on detection accuracy. The findings reported in this paper can help in the autonomous detection of plastic contaminants in cotton fields and potentially speed up the mitigation efforts, thereby reducing the amount of contaminants in cotton gins.

1. Introduction

Plastic contamination in U.S. cotton is a prevalent issue and a matter of concern for both ginners and the entire supply chain of the cotton industry. Considering the severity of the problem, the U.S. Department of Agriculture’s Agricultural Marketing Service (USDA-AMS) began implementing new extraneous matter codes (71 and 72) for cotton classification in 2018 [1]. Plastic contaminants in cotton significantly reduce the fiber’s marketable value, lowering the price paid by yarn spinners to growers [2,3,4]. Plastic contaminants in cotton can come from various sources like plastic wraps on round cotton modules, plastic mulch used in crop production, and from other sources at various stages of the supply chain starting in the field [5,6]. Contaminants may come from plastic rubbish, commonly plastic shopping bags, that are carried by the wind from roadsides and then become entangled with cotton plants across cotton fields [7]. During harvesting, mechanical cotton pickers and strippers do not separate this material from the cotton, and it becomes embedded in seed cotton modules. Plastic contaminants often find their way into the processing machinery of cotton gins [2] and become embedded in cotton bales that are ultimately shipped to spinners. Efforts are being made to design a machine vision (MV)-based automated control system at cotton gins to detect and remove plastic contaminants [2]. However, detecting, locating, and removing plastic contaminants in the field before harvest can minimize the amount that ends up at the gin, thereby reducing the requirement for MV detection at that point. The only current option for field removal is human observation, which is too time-consuming, costly, and inefficient to be a viable process. To solve this problem, earlier studies used unmanned aircraft systems (UAS) with a multispectral camera to detect and locate shopping bags in cotton fields [6,8]. A classifier based on a classical machine learning (ML) algorithm was trained with image-based spectral and textural features of plastic shopping bags [6]. The method resulted in a detection accuracy of over 64% in a cotton field before defoliation. In addition to having inadequate detection accuracy (90% or better is desirable), this method was time-consuming, requiring field data collection and offline image processing, meaning the plastic bags could only be located after a day or so when the images had been processed with the developed algorithm.

Traditional approaches of object detection [6,9] involve feature extraction based on either histogram analysis—such as Histogram of Oriented Gradient (HOG) [10,11] and Edge Orientation Histograms (EOH) [12]—or on image textures [13]. Both HOG and EOH count occurrences of gradient orientation in a localized space of an image, while texture-based approaches calculate the spatial variation of image tones depending on statistics like contrast and entropy [14]. Various methods have been tested to detect and identify foreign matter in cotton. For instance, Himmelsbach et al. [4] used Fourier transform infrared spectroscopy to detect contaminants in cotton fibers. Foreign particles in cotton fiber were identified based on differences in absorbance over a range of wavelengths. This kind of approach is practical for laboratory settings but not in the field because detection with this method requires controlled conditions and is slow and expensive.

Deep learning (DL), on the other hand, can detect objects quickly in the field with good accuracy, which is desirable for efficient contamination removal in the field [15,16]. DL based on remotely sensed images is an attractive approach to achieve near real-time detection and location of plastic bags in cotton fields with improved detection accuracy. DL-based algorithms have been successively used for object detection tasks in many agricultural applications [15,16,17,18,19]. Among many DL-based algorithms, You Only Look Once version 5 (YOLOv5) is a state-of-the-art CNN architecture for real-time object detection. Similar to its earlier versions, YOLOv5 is a one-stage detector in which regression is used for detection and localization, but it is much faster and more accurate than previous versions, with inference speeds of up to 110 frames per second (FPS) [20,21] and 142 FPS [22] as compared to the older versions’ average inference speed of 33 FPS [23,24]. Inference speed varies depending on hardware, image quality, and the size and number of objects present in the images. The end goal of this research is to deploy a trained model for real-time plastic bag detection, which requires high inference speeds; therefore, YOLOv5 was selected for this study. Until the release of YOLOX in 2021 [25], YOLOv5 was the latest version of the YOLO family of object detection algorithms. It has four variants: YOLOv5s, YOLOv5m, YOLOv5l, and YOLOv5x, where s, m, l, and x represent small, medium, large, and extra-large versions, respectively, in terms of network depth and number of parameters. The values for the original and scaled-down versions are shown in

Table 1. Original and scaled-down variants of YOLOv5.

	YOLOv5-Original		YOLOv5-Scaled-Down
	Layers	Parameters	Layers	Parameters
Small (s)	283	7,066,239	224	7,056,607
Medium (m)	391	21,060,447	308	21,041,679
Large (l)	499	46,636,735	392	46,605,951
Extra Large (x)	607	87,251,103	476	87,210,542

.

In the current study, the four scaled-down versions were used. The objectives of this study are as follows: (i) to determine how well YOLOv5 can be used to detect plastic shopping bags in a cotton field; (ii) to determine how the color of plastic shopping bags affects average precision (AP) and accuracy; (iii) to determine how YOLOv5-variant (s, m, l, and x) affects AP, accuracy, mean average precision (mAP@50), and inference speed for different color bags; (iv) to identify an optimal YOLOv5-variant based on a desirability function dependent upon AP for brown and white bags, mAP@50, accuracy, and inference speed; (v) to determine the effect of height of plastic bags on cotton plants on the overall detection accuracy of the most desirable YOLOv5-variant.

2. Materials and Methods

2.1. Experiment Site

To mimic the natural occurrence of grocery store plastic shopping bags in cotton fields, we manually tied a total of 180 plastic bags, 90 white and 90 brown, on cotton plants in a cotton field (26°9′51.62″ N, 97°56′29.66″ W) located near Weslaco in Hidalgo County, Texas (Figure 1). The bags were tied at three different heights (top, middle, and bottom) on the plants before defoliation. The order in which the bags were tied to plants followed a randomized sequence generated by Microsoft Excel software (Microsoft Corporation, Redmond, Washington, DC, USA). The field had been planted with cotton seed of the Phytogen 350 W3FE variety (CORTEVA agriscience, Wilmington, DE, USA). The soil at the experimental site includes two types: Hidalgo sandy clay loam and Raymondville clay loam [26].

2.2. Data Acquisition

A three-band (RGB: red, green, and blue) FC6360 camera (Shenzhen DJI Sciences and Technologies Ltd., Shenzhen, China) integrated on a DJI Phantom P4 quadcopter (Shenzhen DJI Sciences and Technologies Ltd., Shenzhen, China) was used to collect aerial images of the test field from an altitude of 18.28 m (60 feet) above ground level (AGL). Pix4DCapture (Pix4D S.A., Prilly, Switzerland) software was used on a regular smartphone running iOS version 13.5 (Apple Inc., Cupertino, CA, USA) to control the quadcopter in flight. The camera has an image resolution of 1600 × 1300 pixels. The 71 images collected had a ground sampling distance (GSD) of approximately 1 cm/pixel (0.40 inch/pixel). Five ground control points (GCPs) were used for georeferencing aerial images by placing four of them at the corners of the field and one in the middle (Figure 1). An EMLID Reach RS+ (Emlid, Hong Kong, China) GPS unit was used to collect GPS coordinates of the center of the five GCPs. Data were collected on 23 July 2020, between 10 a.m. and 2 p.m. Central Standard Time (CST).

2.3. Image Data Preparation

After the aerial images were collected, they were used to generate an orthomosiac with Pix4DMapper 4.3.33 software (Pix4D S.A., Prilly, Switzerland) which were georeferenced using the GPS coordinates of the five GCPs (Figure 1). The orthomosiac was used only as an aid in visually locating areas of interest on the ground. Individual RGB images of 1600 × 1300 pixels were split into 416 × 416 pixels with Python version 3.8.10 (Python Software Foundation, DE, USA) script [27]. Image augmentation techniques were subsequently applied to increase the number of images in the dataset with the Python library developed by Bloice et al. [28]. The following operations, along with the corresponding probabilities, were used in the augmentation pipeline: rotate with a probability value of 0.7, flip_left_right with a probability of 0.4, zoom_random with a probability of 0.4, percentage_area of 0.8, and flip_top_bottom with a probability of 0.4. These probability values determined the chance of applying the operation each time an image passed through the augmentation pipeline. Once an operation was chosen to be applied, parameter values were randomly applied from within the set range. A total of 1000 samples were generated in each of the 5 iterations, resulting in a total of 5000 augmented images. However, many of these images did not contain either white or brown plastic bags. All such images were discarded, and only the images that contained at least one bag were used, resulting in a total of 1410 images. Then, 10 sets of image data were allocated, with each set containing 141 images. Each of the 10 datasets was divided into training, validation, and testing data in the ratio of 15:3:2. The LabelImg V-1.8.0 [29] software tool was used for annotating ground truth bounding boxes for the two classes, white bags (wb) and brown bags (bb). All these processes can be seen in a workflow pipeline flowchart in Figure 2.

2.4. YOLOv5 Architecture

All four versions of YOLOv5 have similar architectures that vary in depth and network parameters. To understand the function of YOLOv5, one can consider the architecture of YOLOv5s, shown in Figure 3, as generated by the visualization software, Netron version 4.8.1 [30]. The architecture has a total of 25 nodes, also known as modules [31], which are named model/0 to model/24. The first 10 modules form the backbone network, the next 14 modules form the neck network, and the last module forms the head/detection network of the architecture (Figure 3). The backbone and neck network collectively form the feature extractor, while the head/detection network forms the predictor or detector. In model/24, i.e., detection/head network, the value of nc = 2 represents the number of classes the YOLOv5 network was customized to detect.

YOLOv5 is a single-stage detector that uses a “hard-swish” activation function that makes it faster to train as well as to detect [15]. “Hard-swish”, i.e., h-swish, is a modified version of the “swish” activation function that is mathematically defined by Equations (1) and (2) [32,33].

$$\mathrm{swish}\left(\mathrm{x}\right) = \mathrm{x} \mathsf{\sigma }\left(\mathrm{x}\right)=\frac{\mathrm{x}}{1 + {\mathrm{e}}^{-\mathrm{x}}}$$

(1)

$$\mathrm{h-swish}\left(\mathrm{x}\right)=\mathrm{x}\frac{\mathrm{ReLU}6(\mathrm{x}+3)}{6}$$

(2)

2.5. YOLOv5 Training, Performance Metrics, and Desirability Function

The source code for YOLOv5 was obtained from the Ultralytics Inc. GitHub repository of YOLOv5 [34]. Selected model hyperparameters are shown in

Table 2. YOLOv5 hyperparameters used for this study.

Hyperparameters	Values
Learning Rate	0.01
Learning rate Decay	0.2
Momentum	0.937
Weight Decay	0.0005
Batch Size	4
Training Epoch	250

. The PyTorch framework (Facebook AI Research Lab, Menlo Park, CA, USA) with torch version 1.9.0 and Compute Unified Device Architecture (CUDA) version 10.2 (NVIDIA, Santa Clara, CA, USA) were used to implement YOLOv5. The auto anchoring function of YOLOv5 generated 4.5 anchors per target object based on the annotated ground truth bounding boxes (BBs). Original values from the source code were used for enhancement coefficients of hue (0.015), saturation (0.7), and lightness (0.4). Stochastic gradient descent (SGD) was used as the optimization algorithm for each version. SGD is a method of minimizing an objective function during model training by randomly choosing a single data point from a batch of data points, thereby reducing computation time to speed up the training process [35]. Google Colaboratory Pro (Google LLC, Mountain View, CA, USA) was used to train YOLOv5 on Tesla P100 GPU-16GB (NVIDIA, Santa Clara, CA, USA). All four versions of YOLOv5 were trained for 250 iterations with a batch size of 4, using 106 training, 21 validation, and 14 test images, resulting in a total step size of 6625. YOLOv5 was pre-trained on COCO datasets for 80 different classes [36]. The pre-trained weights were used as the starting point, and YOLOv5 was customized for 2 classes based on a custom dataset. Each trained model was saved in pt format and later used for detection.

To evaluate the trained models on each of the ten datasets, precision (P), recall (R), average precision (AP), mean average precision (mAP), accuracy, and F1-score were used as performance metrics. These metrics were calculated based on generalized intersection over union loss (GIoU_loss), confidence over objectness loss, and classification over probability loss. The equations to calculate these metrics are given below.

$$P=\frac{\mathit{TP}}{\mathit{TP}+\mathit{FP}}$$

(3)

$$\mathit{R}=\frac{\mathit{TP}}{\mathit{TP}+\mathit{FN}}$$

(4)

$$\mathrm{Accuracy}=\frac{\mathit{TP}+\mathit{TN}}{\mathit{TP}+\mathit{TN}+\mathit{FP}+\mathit{FN}}$$

(5)

$$\mathit{F1-score}=2\frac{\mathit{P}\ast \mathit{R}}{\mathit{P}+\mathit{R}}$$

(6)

$$\mathit{AP}=\sum _{\mathit{k}=0}^{\mathit{k}=\mathit{n}-1}\left[\mathit{R}\left(\mathit{k}\right)-\mathit{R}\left(\mathit{k}+1\right)\right]\times \mathit{P}\left(\mathit{k}\right)$$

(7)

$$\mathrm{mAP}=\frac{1}{\mathit{n}}{\sum }_{\mathit{k}=1}^{\mathit{k}=\mathit{N}}{\mathit{AP}}_{\mathit{k}}$$

(8)

where N is the number of classes (N = 2 in our case for wb and bb), TP is the number of true positives, TN is the number of true negatives, FP is the number of false positives, FN is the number of false negatives, AP_k is the AP of class k, and n is the number of thresholds (n = 1 in our case). Precision is a measure of TP over predicted positives, while recall is a measure of TP rate and is also sometimes called sensitivity. Accuracy, on the other hand, is an overall measure of true detection. F1-score is the harmonic mean of P and R, i.e., reciprocal of the arithmetic mean of reciprocals of P and R. All these metrics are determined based upon different losses that are calculated as shown below.

$${\mathrm{GIoU}}_{\mathrm{loss}}={\mathsf{\lambda }}_{\mathrm{coord}}\sum _{\mathrm{i}=1}^{{\mathrm{s}}^2}\sum _{\mathrm{j}=1}^{\mathrm{B}}{{\mathrm{l}}_{\mathrm{ij}}}^{\mathrm{obj}} {\mathrm{L}}_{\mathrm{GIoU}}$$

(9)

Confidence Loss/Objectness Loss=

$${\sum }_{\mathrm{i}=1}^{{\mathrm{s}}^2}{\sum }_{\mathrm{j}=1}^{\mathrm{B}}{\mathrm{l}}_{\mathrm{ij}}^{\mathrm{obj}}({\mathrm{C}}_{\mathrm{i}}- {\widehat{\mathrm{C}}}_{\mathrm{i}}{)}^2+ {\mathsf{\lambda }}_{\mathrm{nobj}} {\sum }_{\mathrm{i}=1}^{{\mathrm{s}}^2}{\sum }_{\mathrm{j}=1}^{\mathrm{B}}{\mathrm{l}}_{\mathrm{ij}}^{\mathrm{obj}}{({\mathrm{C}}_{\mathrm{i}}-{\widehat{\mathrm{C}}}_{\mathrm{i}})}^2$$

(10)

Classification/Probability Loss=

$${\sum }_{\mathrm{i}=1}^{{\mathrm{s}}^2}{\mathrm{l}}_{\mathrm{ij}}^{\mathrm{obj}}{\sum }_{\mathrm{c}\in \mathrm{class}}({\mathrm{p}}_{\mathrm{i}}\left(\mathrm{c}\right)-{\widehat{\mathrm{p}}}_{\mathrm{i}}\left(\mathrm{c}\right))$$

(11)

In Equation (9), λ_coord is the penalty coefficient associated with the central coordinate of the predicted bounding box (BB), s² is the number of grid cells generated on the input image, and B is the total number of predefined BB, i.e., the anchors, which is equal to 9 (3 at each scale). In Equations (10) and (11), the existence and absence of an object in the predicted BB are represented by ‘obj’ and ‘nobj’, respectively. Similarly, $C_i$ represents the class of the predicted object, ${\widehat{C}}_i$ represents the class of the ground truth object, and λ_nobj represents the penalty coefficient associated with the confidence loss. Even though these metrics are widely used for evaluating the performance of the trained model and its associated classifier, one must be careful about two important aspects: the way data are sampled and the balance between different classes in sampled datasets. Xie and Braga–Neto [37] found that precision as a performance metric may not be reliable and can be severely biased under separate sampling conditions. The sampling technique used in this research was kept consistent to avoid bias induced by sampling discrepancy. Similarly, Luque et al. [38] showed that an imbalance in the proportion of instances between positive and negative classes could affect classification accuracy, and they proposed geometric mean (GM) or Bookmaker Informedness as the most bias-free metrics. To solve this issue, YOLOv5 also uses focal loss [39].

In addition to evaluating each model’s performance separately, the best model had to be identified among all four variants (s, m, l, and x) based on the performance metrics and inference time. Identifying the best overall model would enable its deployment on a UAS using CV for real-time plastic bag detection in a cotton field. Since this problem is similar to optimization with multiple response variables, the desirability function was used to convert all the response variables into a single one by first scaling them from 0 to 1, then taking their geometric mean (GM), and finally, combining them as shown in Equation (12) [40,41].

$$\mathrm{D}=\sqrt[\mathrm{p}]{({\mathrm{d}}_{1 }\times { \mathrm{d}}_2 \times { \mathrm{d}}_{3 }\times \dots {\mathrm{d}}_{\mathrm{p}})}$$

(12)

Here, D is the overall desirability of all the p response variables (in our case, p = 4) denoted by d₁,d₂ …d_p. In our case, d₁ = mAP, d₂ = accuracy of white bags, d₃ = accuracy of brown bags, and d₄ = inference speed. To implement this function, JMP Pro version 15.2.0 software was used (SAS Institute, Cary, NC, USA). In this study, the response goals for mAP and accuracies for white and brown bags were set to 1, 0.85, and 0.8 for high, middle, and low, respectively, and then the desirability was maximized using constrained Newton’s method as explained by Dennis and Schnabel [42].

2.6. Experiment Design

To achieve the second objective, a fixed effect model was designed to examine the effect of the color of plastic shopping bags on AP and accuracy with all four variants of the YOLOv5 model. A total of 10 observations were made by randomly selecting 10 sets of 14 test images containing both white and brown plastic shopping bags and then attempting to detect them with the trained YOLOv5s, YOLOv5m, YOLOv5l, and YOLOv5x models. Assuming Y_ijc is the j-th observation from treatment group ic, where ic = W, B, and jc= 1,2,3, …10, the fixed effect model can be expressed as:

$${\mathrm{Y}}_{\mathrm{ijc}}={\mathsf{\mu }}_{.\mathrm{c}}+{\mathsf{\alpha }}_{\mathrm{ic}}+{\mathsf{\epsilon }}_{\mathrm{ijc}}$$

(13)

Here, µ._c is the average of all the treatment group means, while ɛ_ijc is the error, or the residual term, for the fixed effect model, and α_ic is the i-th treatment effect. The null and alternative hypotheses for this model can be shown as

$${\mathrm{H}}_{0\mathrm{c}}:{\mathsf{\alpha }}_{\mathrm{W}}={\mathsf{\alpha }}_{\mathrm{B}}$$

(14)

$${\mathrm{H}}_{1\mathrm{c}}:{\mathsf{\alpha }}_{\mathrm{W}}\ne {\mathsf{\alpha }}_{\mathrm{B}}$$

(15)

The first step was to test the normality assumption with the Shapiro–Wilk test at the 95% confidence level (α = 0.05) with Python’s SciPy Stats module [43]. Then, the model effect test was conducted with the Standard Least Squares (SLS) personality fit with JMP Pro version 15.2.0 software (SAS Institute, Cary, NC, USA). The SLS method is used to fit linear models for continuous response variables using the method of least-squares.

To achieve the third objective, a fixed effect model was designed to test the effects of YOLOv5 versions on the AP, mAP, and inference speed for white and brown bags. Again, there were 10 observations, 4 treatment groups (YOLOv5s, YOLOv5m, YOLOv5l, and YOLOv5x), and 4 output metrics (AP for white bags, AP for brown bags, mAP, and inference speed). Assuming Y_ij is the j-th observation from treatment group I, where i = s, m, l, x, and j = 1,2,3, …10, the fixed effect model can be expressed as

$${\mathrm{Y}}_{\mathrm{ij}}={\mathsf{\mu }}_{.}+{\mathsf{\alpha }}_{\mathrm{i}}+{\mathsf{\epsilon }}_{\mathrm{ij}}$$

(16)

Here, µ. is the average of all the treatment group means, while ɛ_ij is the error (the residual term) for the fixed effect model, and α_i is the i-th treatment effect. The null and alternate hypotheses for this model can be shown as

$${\mathrm{H}}_0: {\mathsf{\alpha }}_{\mathrm{s}}={\mathsf{\alpha }}_{\mathrm{m}}={\mathsf{\alpha }}_{\mathrm{l}}={\mathsf{\alpha }}_{\mathrm{x}}$$

(17)

$${\mathrm{H}}_1:{\mathsf{\alpha }}_{\mathrm{s}}\ne {\mathsf{\alpha }}_{\mathrm{m}} \mathrm{or} {\mathsf{\alpha }}_{\mathrm{s}}\ne {\mathsf{\alpha }}_{\mathrm{l}} \mathrm{or} {\mathsf{\alpha }}_{\mathrm{s}}\ne {\mathsf{\alpha }}_{\mathrm{l}} \mathrm{or} {\mathsf{\alpha }}_{\mathrm{m}}\ne {\mathsf{\alpha }}_{\mathrm{l}} \mathrm{or} {\mathsf{\alpha }}_{\mathrm{m}}\ne {\mathsf{\alpha }}_{\mathrm{x}} \mathrm{or} {\mathsf{\alpha }}_{\mathrm{l}}\ne {\mathsf{\alpha }}_{\mathrm{x}}$$

(18)

Like the previous case, the normality assumption was first tested, and then the model effect, with SLS personality fit with JMP Pro version 15.2.0 software (SAS Institute, NC, USA). Finally, to choose the optimal YOLOv5 model, the desirability function in the prediction profiler graphs obtained from the YOLOv5 model type effect tests was used.

Once the most desirable variant of YOLOv5 was chosen, a fixed effect model test was performed to determine the effect of the height of plastic bags (top, middle, and bottom) on the overall detection accuracy by the trained YOLOv5 model. For these, 10 random image samples (416 × 416 pixels) were chosen, and the most desirable variant of the trained YOLOv5 model was used for the detection of plastic bags. The number of detected bags was matched visually with the experiment layout to determine the number of top, middle, and bottom bags detected out of the total top, middle, and bottom bags present in each image. Then, a percentage ratio for each (top, middle, and bottom) was calculated. This is called bag-based accuracy hereon.

3. Results

3.1. Overall Performance of YOLOv5 for Plastic Bag Detection in Cotton Field

Figure 4 shows the three types of losses that were calculated and plotted for training and validation datasets.

The plot titles with val_ as the prefix were obtained from the validation dataset. Based on the 40 total observations (10 observations from 4 variants of YOLOv5), the mean AP for white bags was 90.53% with a standard deviation of 0.08. The mean AP for brown bags was 84.84% with a standard deviation of 0.06. The mean of mAP was found to be 87.68% with a standard deviation of 0.05. The accuracy for white bags had a mean of 92.35% and a standard deviation of 0.05. The mean accuracy for brown bags was 77.87% with a standard deviation of 0.11. The mean inference speed was 81.43 FPS with a standard deviation of 39.78. Figure 5A is a precision-recall plot obtained after training YOLOv5s with one of the 10 observation datasets. The areas under these curves can be used to determine AP for white bags (wb) and brown bags (bb), as well as mAP corresponding to both. Figure 5B includes plots of the F1-score against confidence values, which are directly proportional to a confidence value of around 0.33. After that point, the F1-score starts to decrease with increasing confidence values, indicating that the predicted BBs can be associated with more confidence about the presence of included objects but also that they are less precise and accurate for predicted classes (wb and bb). Until the confidence value of approximately 0.62, the predicted BBs are more accurate, precise, and sensitive about wb, but after this point, they become more precise, sensitive, and accurate about bb. Figure 6 is a confusion matrix heat map in which white bags have a higher true positive rate than brown bags. Figure 7 includes some results of the trained model used for detection on the test dataset.

3.2. Color Effect Tests

When the color effect model was tested on the four versions of YOLOv5, the resulting histogram and density plots (Figure 8, Figure 9, Figure 10 and Figure 11) showed the distribution of AP and accuracy for each version (s, m, l, and x) as well as side-by-side box plots with lines connecting their means and medians along with pair-wise Student’s t-test values. Based on 10 observations for both white and brown bags, the mean of AP values for s, m, l, and x models were 89.85%, 90.15%, 87.94%, and 82.81% with standard deviations of 0.06, 0.05, 0.07, and 0.11, respectively. The overall mean accuracies for s, m, l, and x were 84.80%, 88.95%, 85.85%, and 80.85%, respectively. Based on colors, both the AP and accuracy for white bags were significantly higher than the brown bags for s and l (Figure 8 and Figure 10). However, for m and x, only the accuracy for white bags was significantly higher than the brown bags (Figure 9 and Figure 11). The AP and accuracies were expected to be higher with increasing size and depth of the model, but this was true only from s to m, not from m to l or l to x. A possible reason for this could be that there were fixed numbers of training iterations for all versions. Hence, the l and x versions may not have reached convergence during the 250 iterations. In this study, it was necessary to fix the hyperparameter values for consistency in results comparisons.

Table 3. Plastic bag color effect test on average precision and accuracy of different variants of YOLOv5.

Model	Average Precision		Accuracy
	F Ratio	Prob > F	F Ratio	Prob > F
YOLOv5s	5.7812	0.0272	19.2649	0.0004
YOLOv5m	3.0022	0.1002	6.6480	0.0189
YOLOv5l	12.1663	0.0026	19.8409	0.0003
YOLOv5x	0.9136	0.3518	21.6296	0.0002

shows the results of the effect of the color of plastic bags on AP and accuracy for all four model versions. There was evidence to determine that color had a significant effect (α = 0.05) on accuracy for all four versions. Similarly, there was evidence to determine that color had a significant effect on AP for s and l but not for m and x at the 5% significance level. From the boxplots of accuracy as a function of the color of the bags, white bags had significantly higher accuracy than brown bags (Figure 12).

A chart showing difference inference times that were obtained by testing 14 images during each of the 10-test datasets.

3.3. Model Type Effect Tests

Values of AP, accuracies, mAP, and inference times for the model effect test are given in

Table 4. YOLOv5 Model type effect test on AP, accuracy, and mAP for white and brown color plastic bags.

Model	F Ratio	Prob > F	White Bags		mAP		Brown Bags
			F Ratio	Prob > F	F Ratio	Prob > F	F Ratio	Prob > F
AP			2.15	0.11	4.63	0.01	3.27	0.03
Accuracy			0.44	0.73			3.66	0.02
InferenceTime	27,665.3	<0.0001

. The model type did not have a significant effect on accuracy and AP for white bags, but it did have a significant effect in all the other metrics (α = 0.05). In part, this means that the detection of white bags was not affected by the choice of model, whereas the detection of brown bags was affected by the type of model used. Furthermore, inference time (i.e., detection speed) depends on the choice of model regardless of bag color. This result was expected because larger models are deeper with many more parameters that require more processing time, as evidenced by the different inference times obtained for each observation test dataset and for all four versions (Figure 13). The distribution of mAP for the four YOLOv5 version models can be seen in Figure 14, with histogram and density plots as well as side-by-side box plots with means and medians connected by lines. Pair-wise Student’s t-test values are also shown with the boxplots. The most consistent distribution can be seen for YOLOv5m, but there was an outlier. The mean and median lines are almost parallel between YOLOv5s and YOLOv5m, indicating similar performance. The lines decline from YOLOv5m to YOLOv5l and from YOLOv5l to YOLOv5x, indicating a sequential decrease in the mAP for the larger models. As mentioned earlier, this result can be attributed to the fact that the larger models were likely not trained adequately for their losses to converge. Mean accuracy increased from s to l but decreased from l to x; however, the decrease was not significant. The model type did not have a significant effect on overall accuracy except between m and x (Figure 14).

3.4. Desirability Function as Optimization Method

The most important of the six metrics (AP and accuracy for white and brown bags, mAP, and inference time) were the inference speed, accuracy, and mAP, because the goal was to find an optimal YOLOv5 model with the greatest detection accuracies for white and brown plastic bags at a higher inference speed that can be deployed on a GPU for near real-time detection. The maximum desirability obtained was nearly 0.95 for YOLOv5m, which had the maximum inference speed of 86 FPS, with accuracies for white and brown bags greater than 92% and 85%, respectively, and mAP greater than 90% (Figure 15).

3.5. Plastic Bag Height Effect Test

Once the most desirable variant of YOLOv5 was found, i.e., YOLOv5m, it was used to test how the height of plastic bags on cotton plants affects its detection accuracy (bag-based detection accuracy). Bag-based detection accuracies, i.e., the percentage of top, middle, and bottom bags that were detected in each of the ten test images, are shown in

Table 5. Effect test of plastic bag height on bag-based detection accuracy. Top, middle, and bottom show the percentage of corresponding bags detected in each of the ten test images on which the trained YOLOv5m model was used.

Image_Sample_Number	Top	Middle	Bottom
1	62.5	12.5	0
2	100	0	0
3	100	100	0
4	100	0	0
5	100	0	0
6	100	33.33	0
7	100	100	50
8	100	50	0
9	100	100	0
10	80	100	0

.

It was found that, on average, 94.25% of the top bags (white and brown), 49.58% of the middle bags (white and brown), and 5% of the bottom bags (white and brown) were detected by the trained YOLOv5m model. A fixed effect model test determined that the height of plastic bags had a significant effect (p < 0.0001) on bag-based detection accuracy.

4. Discussion

4.1. Overall Performance of YOLOv5

Results from this study showed that YOLOv5 was able to detect plastic shopping bags in aerial images of a cotton field with an overall accuracy of 92% for white bags, 78% for brown bags, and mAP of 88%. This means that using the deep learning method, detection accuracy for white bags improved by nearly 19%, for brown bags by 18%, and the overall detection accuracy improved by 24% when compared to the previous methods used [6]. Apart from higher detection accuracy, this method was also able to perform detection at speeds that make it practically viable for near real-time detection (Figure 13). Therefore, this approach can be used to potentially enable field robots like unmanned ground vehicles to remove the bags autonomously, as proposed by Hardin et al. [44]. It was also found that the overall detection accuracy of white bags was significantly higher than that of brown bags. This result was expected because the brown bags had lower contrast with their surroundings, particularly those that were tied at the bottom of the plants and looked like the soil in the background. This is in agreement with the findings by Liu et al. [45], in which detection of aircraft by YOLOv5 decreased due to the white background, as most of the aircraft are white in color.

It is known that the deeper the CNN network, the better the performance. Deeper networks learn better [46,47], explaining why, in most cases, the performance of YOLOv5x > YOLOv5l > YOLOv5m > YOLOv5s [22]. Results of our study showed this to be true for YOLOv5s and YOLOv5m, but the performance was found to be less for YOLOv5l and YOLOv5x. There can be multiple reasons for this, one of which may be attributed to the fact that the larger and deeper models were not trained enough to enable the losses to converge within the fixed 250 iterations. DL and ML models are stochastic in nature; hence fine-tuning hyperparameter values of the l and x other than the ones in Table 1 can result in different and perhaps higher detection results. However, for a fair comparison, all four versions were trained with the same number of iterations and fixed values of hyperparameters, as explained in Section 5 and Section 6.

4.2. Effect of Color of Bags on YOLOv5 Performance

From this study, it was found that the color of plastic bags had a significant effect on overall detection accuracy for all four variants of YOLOv5. White bags were always detected with higher accuracy than brown bags, as was expected due to their higher contrast with the background. This result could be different if the cotton plants had many open bolls, thus creating a white background and making the detection of white bags difficult due to their decreased contrast with the background. An interesting finding was that bag color had a significant effect on AP for YOLOv5s and YOLOv5l but not for the YOLOv5m and YOLOv5x. This means that the m and x models were fine-tuned and trained such that the AP of these models was robust to the changes in the color of plastic bags.

4.3. Effect of Model Type

Findings from this study indicated that YOLOv5m performed the best in terms of overall accuracy and mAP followed by YOLOv5s, YOLOv5l, and YOLOv5x. However, no significant effect was found on the overall detection accuracy of plastic bags except between YOLOv5m and YOLOv5x. These results suggest that if any of the models are trained adequately, they can be used for plastic bag detection in cotton fields without major concerns about detection accuracy. This is only true if computation cost is not an issue, which in practical situations is always a matter of concern.

4.4. Optimal Model

Computation cost in terms of time, energy to operate the system, money, etc., has always been a concern when deploying CV algorithms on edge computing devices [48,49]. In our study, we focused on finding the optimal YOLOv5 model out of the four variants, keeping the detection accuracies for white and brown bags at least 80% and inference speed of at least 35 FPS. The desirability function, as explained in Section 2.5, was helpful in determining the most practically viable model for near real-time detection of plastic shopping bags in cottonfields, which is YOLOv5m.

4.5. Effect of Plastic Bag Height

The bag-based detection accuracies were found to be relatively consistent for the top and bottom bags, with a standard deviation of 12.80 and 15.81, respectively; however, in the case of middle bags, it was highly inconsistent, with a standard deviation of 46.17. On average, the trained YOLOv5m was able to detect top bags with significantly higher detection accuracy than the bottom bags. This is as expected because most of the bags at the bottom remained hidden under the canopy of cotton plants and, therefore, could not be seen in the aerial images collected by the UAS. The middle bags were detected at higher accuracy than the bottom bags but at a significantly lower value than the top bags.

4.6. Limitations

The findings of this study are limited by the fact that fixed hyperparameters and training iterations were used for all YOLOv5 variants, such that deeper models like YOLOv5l and YOLOv5x apparently failed to converge their losses. These models would likely perform better if trained for a greater number of iterations. Furthermore, the inference speed reported here is for a GPU that is part of a high-performance computing system that is not readily available for practical applications in field conditions. Hence, the inference speed may be significantly different for a field-based computing platform. In addition, this study reports the findings based on images collected from 18.28 m (60 feet) above ground limit (AGL), and therefore, the results may improve when high-resolution images are collected from either lower elevation or using a high-resolution camera or combination of both.

5. Conclusions

In conclusion, this paper demonstrates the successful utilization of YOLOv5 for detecting white and brown plastic shopping bags in a cotton field using RGB aerial images obtained from UAS. The average accuracies achieved were 92.35% and 77.87% for white and brown bags, respectively, with an mAP of 87.68%. Compared to classical ML methods previously employed, YOLOv5 exhibited a 24% higher overall accuracy in bag detection. The study also highlights the significant influence of bag color and height on cotton plants on detection accuracies, concluding that YOLOv5m is the most practical model for near real-time detection. Furthermore, the analysis reveals that top bags were detected with significantly higher accuracy (94.25%) compared to middle (49.58%) and bottom bags (5%). Additionally, it was observed that the background color had a detrimental effect on the detection accuracy of plastic shopping bags. Consequently, it is recommended to survey cotton fields before the open boll stage, during the period when the leaves are green and the cotton bolls are closed, to ensure that the aerial images are free of white and brown backgrounds that may hinder YOLOv5’s performance. Moreover, it is advised to conduct surveys when the plant canopies are not overly dense to avoid overlooking bags entangled at the bottom of cotton plants, which often remain undetected and may require manual scouting.

6. Future Work

Since YOLOv5 is built upon a PyTorch (Facebook AI Research Lab, Menlo Park, CA, USA) framework, which is written in Python programming language, it is easy to deploy on embedded systems like GPUs. The inference results (Figure 9) are promising for real-time object detection. However, these were obtained on Tesla P100 GPU -16GB (NVIDIA, Santa Clara, CA, USA), which provides computation speed not possible to deploy at “the edge” (i.e., at the source of image data generation, a UAS with camera and GPU). The goal of this research is to deploy the developed AI model on a low-cost and lightweight edge computing device that can be easily mounted on a UAS. Developing near real-time computation on an edge computing device will be part of future research. The future work will also involve the extraction of central coordinates of detected BBs with plastic shopping bags and then georeferencing them to a geodetic coordinate system that might allow ground-based robots to collect the detected plastic shopping bags. Ideally, detection and coordinate extraction will be performed in near real-time with the trained YOLOV5m model on a low-cost, lightweight GPU device mounted along with a camera on a UAS.