Enhanced Weed Detection in Sustainable Agriculture: A You Only Look Once v7 and Internet of Things Sensor Approach for Maximizing Crop Quality ^†

Abstract

Effective weed detection is essential in modern agriculture to improve crop yield and quality. Farmers can optimize their weed control strategies by applying herbicides tailored to the identified weed species and the areas they affect. Real-time object detection has been transformed by recent advances in image detection technology, especially the You Only Look Once (YOLO) algorithm; of these, YOLOv7 has been shown to be more accurate than its predecessors in weed detection. Because of its novel E-ELAN layer, the YOLOv7 model achieves 97% accuracy, compared to the estimated 78% accuracy of the YOLOv5 model. This study suggests using Internet of Things (IoT) sensors in conjunction with YOLOv7 to improve weed detection using an integrated strategy. It was advantageous to include a variety of sensors in the proposed method to detect and manage weeds. Greater accuracy and comprehensiveness were achieved by combining a variety of sensors to improve the data obtained. An enhanced weed detection system has been developed by utilizing each sensor type’s distinct information. A comprehensive set of environmental data, including soil moisture, temperature, humidity, light intensity, pH, and ultrasonic distance sensors, were used to determine correlation with weed growth patterns. This information was sent to a central Internet of Things gateway for in-the-moment analysis and merged with video footage of agricultural fields. Farmers will be able to anticipate weed infestations and optimize their management tactics using predictive analytics, which will be made possible by integrating sensor data with YOLOv7’s weed-detecting capabilities. This methodology seeks to revolutionize weed control procedures by utilizing cutting-edge technology and IoT connectivity, making them more effective and efficient.

1. Introduction

Weeds are unwanted plants in crops that utilize the nutrients required by the crops to grow. Because of weeds, the crop may not obtain all the sufficient and required nutrients to produce the estimated yield. As the population is growing rapidly, estimated to reach 9 billion people by 2050 [1,2], every government is focusing on doubling food production to meet the projected food demands. To achieve this goal, we must eliminate all the parasitic plants, insectivorous plants and saprophytic plants affecting crop plants. So, farmers are trying to eliminate weeds so that the crop plants obtain the required nutrients.

In the past, weeds were identified manually, which required a lot of time and energy because someone has to physically check every plant. In that process, weeds were sometimes mistaken for actual crops. To address this problem, technology is rapidly being developed [3]. Efficient models to identify weeds can be developed using a few techniques. Through these models, researchers can reduce farmers’ workload.

The statistics provided in [4] show that the cost of weeds in agricultural fields strongly influences the growth rate of crop yield, and the secondary production rate can be seriously reduced due to the diseases caused and transmitted by weeds. It is also found that the fertility rate of the soil is decreased due to unwanted plant species (weeds). Recent developments in image recognition technology, particularly the You Only Look Once (YOLO) algorithm, have transformed real-time object detection in agriculture and improved the current weed detection approaches. Out of all the YOLO versions, YOLOv7 has been proven to be more accurate in identifying cannabis species than YOLOv5, with a 97% accuracy rate against 78% for YOLOv5 as mentioned by the authors in this research [4]. This high degree of precision lessens the likelihood of misidentifying weeds and crops, which is a problem with manual detection techniques.

Furthermore, weed detection can be improved even further by combining Internet of Things (IoT) sensors with the YOLOv7 algorithm. Weed growth patterns are greatly influenced by environmental factors such as soil pH, temperature, humidity, light intensity, and soil moisture. A more thorough picture of the distribution of weeds throughout a field can be attained by utilizing sensors to track these variables in real time. Predictive weed control is made possible by the real-time examination of these data via an IoT gateway. When IoT sensor data are integrated with video footage, farmers are better able to detect possible weed infestations early on, which lowers the costs of management and allows for more accurate herbicide application.

The consideration of ecological and economic sustainability is also emphasized in this approach. The above research has indicated that the incorrect or overuse of herbicides has an adverse effect on agricultural production as well as soil health, which can result in long-term degradation. Farmers could have an affordable solution by using a method that combines IoT sensors with sophisticated weed identification systems like YOLOv7. Farmers can reduce their expenses and increase crop yields by lowering the need for blanket herbicide applications. This is particularly important in areas like Vijayawada, India, where field trials are being carried out with this technique. By using predictive analytics to analyze sensor data, farmers may enhance the profitability and sustainability of their farming methods by making well-informed decisions regarding weed control.

2. Methodology

The proposed system shown in Figure 1 is based on the efficient object detection algorithm. YOLO itself is one of the effective and efficient object detection algorithms. In that particular algorithm, the proposed framework focuses on the YOLOv7 version, which is the most recent version of the YOLO algorithm.

2.1. Data Collection from Sensors

In modern agriculture, gathering environmental data from multiple sensors is essential to significantly improve weed detection and management techniques. Data collection from many sensor types, such as from pH, light intensity, temperature, humidity, soil moisture, and ultrasonic distance sensors, is part of this process. After that, the data are sent to an Internet of Things gateway, which is the main hub for data processing, aggregation, and transfer to the cloud for additional analyses.

Soil moisture sensors: By measuring the volumetric water content of the soil, these sensors can provide information on the amount of moisture present and whether irrigation is necessary to prevent the growth of weeds. Soil moisture sensors typically measure the dielectric constant of the soil, which changes with its water content.

Temperature sensors: These sensors aid in studying the environmental factors that promote the growth of weeds by measuring the temperature of the soil and the surrounding air. Digital temperature sensors like thermistors or thermocouples can convert temperature readings into an electrical signal.

Humidity sensors: These instruments gauge the amount of moisture in the atmosphere, which can affect the health of crops and the development patterns of weeds.

Capacitive or resistive sensors are commonly used for measuring atmospheric humidity. Light intensity sensors measure light levels, which are critical for photosynthesis and impact crop and weed growth. Light sensors use photodiodes or phototransistors that measure the intensity of light and convert it into a readable signal.

pH sensors: The pH of the soil affects crops and weeds as well as the availability of nutrients. These sensors support the monitoring of the alkalinity or acidity of the soil. pH sensors typically use an electrode that measures the hydrogen ion concentration in the soil.

Ultrasonic distance sensors: These sensors provide information on the physical characteristics of the field by measuring distances to identify plant heights and weed growth. Ultrasonic sensors typically operate at frequencies above 20 kHz and are used in automated systems for weed detection by scanning fields.

The sensor’s output (analog voltage) can be digitized using an analog-to-digital converter (ADC). The digital data (representing moisture level) can then be transmitted to the gateway using communication protocols like I2C, SPI, or UART.

2.2. Data Collection/Preparation

Two different datasets were used to train and test the object detection algorithm. The primary dataset was a real-time dataset, and the secondary dataset was collected from Kaggle and used in this work.

The primary video dataset was collected using a mobile camera with a 12 MP and 30 fps configuration. The videos in this dataset are 20 s long. The authors went to a nearby crop field to collect the dataset and used a camera to capture the video. The authors selected a particular area in the crop field and gathered footage from that specific area. This field contained the same crop and weeds as the secondary dataset.

The secondary dataset was the crop_weed_BBox dataset, which was collected from Kaggle [5] and contains 1300 images of sesame crops and different labeled weeds. The images in this dataset are a 512×512×3-sized color images. These images are in the YOLO format.

2.3. Synchronized Data Collection and Metadata in Video and Image Files

The system captures video and images of the field where weeds need to be detected. At the same time, the sensor data are collected via the transmission gateway, which provides environmental context from the time when the video/image was captured. The key is timestamping both the video/image data and sensor data so they can be synchronized. The sensor data can be embedded as metadata in a video file format (e.g., MP4, AVI). This can be achieved using video processing tools like FFmpeg, where the sensor data are added as metadata at regular intervals corresponding to the video frames. For example, for every frame or specific time intervals (e.g., every second), the moisture level, temperature, humidity, etc., are included in the video’s metadata. When analyzing the video, the sensor data can be extracted frame-by-frame to correlate environmental conditions with the visual data.

After video/image capture and sensor data logging, both data sources can be combined into a unified dataset for further analysis or machine learning. This can be conducted by linking frames of the video (or individual images) with sensor readings, as shown in

Table 1. Comparison of various environmental and soil conditions for weed detection.

Frame/Image ID	Moisture Level	Temperature (°C)	Humidity (%)	Light Intensity (lx)	Soil pH	Weed Distance (cm)	Label (Weed/No Weed)
Frame_00001	40%	28	55%	1500	6.5	25	Weed
Frame_00002	42%	29	57%	1600	6.7	22	No Weed

.

A snapshot of sample dataset is provided in Table 1.

Training/Testing the Model

Training and testing are essential yet critical tests in building any predictive model. In this study, the secondary dataset was divided into two parts: 70% of the entire dataset was for the training of the model, and of the remaining 30%, 5% was taken as the validation set and 25% as the testing set.

We imported this model’s code from the official YOLOv7 page [6]. This model was then trained with the training dataset. Then, for the testing, the testing dataset helped with testing the model’s performance and accuracy.

3. Modeling

The most recent version of YOLOv7 exceeded all other object detectors with over 30 FPS on GPU V100 and outperformed them in speed and accuracy, all falling between 5 and 160 frames per second. Regarding real-time item detectors, its AP of 56.8% was the greatest. The transformer-based complete detector SWINL Cascade-Mask R CNN (9.2 FPS A100, 53.9% AP) achieved 50.9% speed and 2% accuracy; however, convolution was inferior to that of the YOLOv7-E6 object detector (56 FPS V100, 55.9% AP), which was 50.9% quicker. With a 8.6 FPS A100 and 55.2% AP, the base detector ConvNeXt-XL Cascade-Mask R-CNN averaged a speed of 55.1% with a 0.7% AP [6].

3.1. Model Architecture

The fully connected neural network (FCNN) of the YOLO architecture is shown in Figure 2. There are three primary parts in the YOLO framework: the neck, head, and backbone. One of the initial training datasets used for the backbone was ImageNet for classification. Generally, recognition is introduced at a lower resolution than for the final recognition model because recognition requires finer detail than categorization. The neck predicts the probability and bounding box coordinates by utilizing the characteristics from the fully connected layers and the convolutional layers of the backbone. The network’s last output layer, the header, can be exchanged for transfer learning with other layers using the same input format [7].

For real-time applications, the YOLO algorithm gave more frames per second and performed better in all aspects considered thus far. Rather than picking interesting regions in an image, the YOLO technique is a regression-based approach that predicts classes and bounding boxes for the entire image in a single run. To understand it fully, we must first comprehend what the YOLO algorithm predicts. The goal was to predict the object’s class and the bounding box that indicates its location. Four descriptors can be used to characterize each bounding box:

(a)

Center of the box (bx, by);

(b)

Width (bw);

(c)

Height (bh);

(d)

Value c corresponds to the class of an object.

There is also a prediction for the actual number, pc, which is the probability that the object is inside the bounding box.

YOLO divides the image into cells (usually a 19 × 19 grid) rather than looking for interesting regions in the input image that may contain objects. Each cell in this predicts the K bounding boxes [8].

An object is considered in a particular cell only if the anchor box’s center coordinates are located in that cell. This property always causes the center coordinates to be calculated relative to the cell, but the height and width are calculated relative to the overall image size.

YOLO determines the probability that a cell contains a particular class during one forward propagation pass. This formula is

score_c, = p_c × c_i

(1)

The most likely class is chosen and assigned to that grid cell. A similar process is performed for all the grid cells present in the image.

This score shows each grid cell before and after predicting the class probabilities. After predicting the class probabilities, the next step is non-maximal suppression. This helps the algorithm to remove unwanted anchor boxes. The number of anchor boxes is calculated based on class probabilities.

To solve this problem, non-maximal suppression removes the bounding boxes that are very close to each other by performing an intersection over union (IoU) on the highest class probability.

The IoU values are computed for all bounding boxes with the highest class probability corresponding to the bounding box, and bounding boxes with IoU values larger than the threshold are rejected. This means these two bounding boxes cover the same object, but the other one is unlikely to be the same, so it is excluded.

Subsequently, the algorithm finds the bounding box with the next highest class probability and performs the same process until only different bounding boxes remain.

The algorithm finally outputs the required vector detailing the bounding boxes for each class.

The most critical parameter of the algorithm, the loss function, is shown below. YOLO learns all four prediction parameters simultaneously (see above). Here, i and j are the two input values.

$${\lambda }_{coord} {\displaystyle \sum }_{i=0}^{S2}{\displaystyle \sum }_{j=0}^B{1}_{ij}^{obj} [{\left(x_i-{x^{\prime }}_i\right)}^{2 }+{\left(y_i-{y^{\prime }}_{i }\right)}^2]$$

where,

B is the number of bounding boxes per grid cell.

${\lambda }_{coord}$ is a weight term to prioritize localization accuracy.

$1_{ij}^{obj}$ is an indicator function that is 1 if an object exists in grid cell I, else 0.

The modeling of YOLOv7 mainly focuses on two features as per the original paper introducing YOLOv7 [6]: extended efficient layer aggregation network (E-ELAN) and model scaling for concatenation-based models.

3.2. Extended Efficient Layer Aggregation Network (E-ELAN)

A highly efficient layer aggregation network mainly considers the parameter range and computational density. The VovNet version (CNN makes DenseNet more efficient by combining all the features in the final characteristic curve as simply as possible) and the CSPVoVNet version improve the input–output channel ratio at the community inference speed, having effects on the element-sensitive operations. YOLOv7 is an extension of ELAN, known as E-ELAN. A significant benefit of ELAN is that deeper communities can explore and converge more effectively by controlling the gradient paths [9].

E-ELAN significantly modifies the structure within the computational block, leaving the structure of the transition layer completely unchanged. We use augmentation, blending, and merging strategies that complement the learning capacity of the community without destroying the gradient paths. The method here involves applying a configuration convolution to expand the channels and ranges of the computational blocks, thereby applying the same configuration parameters and channel multipliers to all computational blocks in the computational layer. Then, the function maps computed in each compute block are shuffled and concatenated. Therefore, the range of channels within each functional mapping organization is likely the same as that within a unique structure. Finally, these companies are merged from the functional map. E-ELAN as mentioned above self learns through various functions.

3.3. Model Scaling for Concatenation-Based Models

The primary purpose of version scaling is to adjust some parameter attributes and generate modes at different scales to improve inference speed. However, implementing these techniques in a concatenation-based forest and shrinking or increasing the depth can lead to simultaneous translational layer growth and intra-layer shrinkage after concatenation, which is mainly due to the complete calculation blocks.

We could not analyze the exceptional scaling factors sequentially but concluded that they should be considered largely as a holistic chain-based version. If the intensity scaling is considered, as an example, such a move can introduce a trade-off between the input and output channels of the transition layer, leading to lower hardware utilization. YOLO v7’s composite scaling technology preserves the version of its preliminary layout and allows it to continue in its first-class form. This is because even if the strength factor of the calculation block is scaled, without adding a new layer.

Calculate the alternatives for the output channels of this block. Then, perform the width factor scaling with the same amount of substitution in the transition layer. This retains the preliminary layout version and the main shape.

3.4. Building the Model

Model building is one of the most important parts for achieving the required results. Model building primarily takes place using the secondary dataset as it is much more convenient for building a good model. The training and testing must be conducted for any model. The training acts as the backbone of the model. The predictions of a model are based only on the training. If a model is trained with well-labeled data, the prediction accuracy will be low, so training should be conducted with some unlabeled data.

After training the model, testing took place. The testing was conducted on 30% of the secondary data. After testing, the data were validated. For validation, model randomly chose the images from the validation set and performed prediction on that particular random image. Every time we ran the model, the photos changed automatically to ensure the model could predict all the images equally. So, this method ensured that the model was appropriate.

As shown in Figure 3, the prediction is performed with bounding boxes; the bounding have numbers 0 and 1 on the top, where 1 indicates that it contains an image of a weed; 0 indicates an image of the crop, without any weeds. To identify the difference, the colors of the bounding boxes are also different: a blue bounding box indicates the crop, and an orange box indicates weeds. This model can predict different crops in one image. It does not give a bounding box for one whole image: it gives a bounding box for both crops and weeds, so that if an image contains more than one weed or more than one crop, they can be discriminated very easily.

3.5. Model Predictions

For evaluating this model, the primary dataset was used as it enabled much more accurate predictions. As the primary dataset was a video dataset, it was a more relevant dataset than a real-word dataset. The data that this model would use in the real world are dynamic data, like video data. Farmers do not only use image data: they can also use video data, so this model was evaluated on real-world data and performed very well and accurately in the real world.

As shown in Figure 4, video is directly used for model predictions. All the pre-prediction is conducted by the model in the background. In this model, there is no need to convert the videos into frames and label them individually. As this model was built on the pre-labeled dataset of the crops and weeds that were in the video, this work was easy.

4. Evaluation Metrics

Prediction metrics such as accuracy, recall, F1 score, and precision were then calculated to evaluate the model in CCP. The true positives, false positives, true negatives, and false negatives were calculated from the confusion matrix [10,11] and are displayed in Table 2. The precision curve represents the degree to which repeated measurements under the same conditions are unchanged [12].

Actual positive: It is a weed.

(a)

Actual negative: It is not a weed

(b)

Prediction is positive: It is predicted as a weed

(c)

Prediction negative: It is predicted as not a weed.

Table 2. Confusion matrix.

	Predicted Positives	Predicted Negative
Actual Positives	True Positives (0.90%)	False Positives (0.02%)
Actual Negatives	False Negatives (0.03%)	True Negatives (0.84%)

Table 2. Confusion matrix.

	Predicted Positives	Predicted Negative
Actual Positives	True Positives (0.90%)	False Positives (0.02%)
Actual Negatives	False Negatives (0.03%)	True Negatives (0.84%)

(a)

True positive (TP): The weed is classified as a weed.

(b)

True negative (TN): The weed is classified as a weed.

(c)

False positive (FP): The crop is classified as a weed.

(d)

False negative (FN): The crop is classified as not a weed.

The model was accurate because, according to the confusion matrix, we found that the true positive and true negative values were larger than the false positives or false negatives.

Accuracy, recall, and precision were interpreted from the above measures. The F1 score was construed from precision and recall.

5. Results and Discussion

The prediction model was employed on the primary dataset. The YOLOv7 model was used in the model because yolov7 very accurately predicted the crops and weeds in the image and video datasets compared with the other models. The model is much less time- and cost-intensive and is easier to implement than the other models. The accuracy for this model was calculated using the F1 score, recall, and precision [13].

Calculating the F1 score is the last step in determining the accuracy of any model. The F1 score is the harmonic mean of accuracy and recall. The precision and recall are combined into one number using the following formula:

Precision = True Positives/(True Positives + False Positives)

Recall = True Positives/(True Positives + False Negatives)

The F1 score considers both precision and recall. This also means that we considered both FPs and FNs.

The first step in calculating the F1 score is producing the precision curve. Precision is the amount of information conveyed in digits. This refers to the resolution or limit of the measurement. Figure 5 shows the precision curve of the proposed model. The precision value was 0.932.

The second step in calculating the F1 score is producing the recall curve. Figure 6 shows the recall curve of our model. The recall is a measure of a model correctly identifying true positives. Thus, for all the images that actually contained weeds, recall indicated how many were correctly identified as having weeds. The recall value obtained in this prediction was 0.99.

After the precision and recall curves were produced, the precision vs. recall curve was calculated, which is shown in Figure 7. The precision vs. recall curve is a direct representation of the precision (y-axis) and recall (x-axis). When the dataset is imbalanced, the number of negative outcomes is much higher than the positive outcomes (or the number of images without weeds is much higher than the number of images with weeds). The precision vs. recall curve value for our model was 0.845.

Precision and recall are the two components of the F1 score. The goal of the F1 score is to combine the precision and recall metrics into a single metric. At the same time, the F1 score was designed to perform well even with imbalanced data; the F1 score results are shown in Figure 8. The F1 score value of our model was 0.78.

F1 = 2 × (precision × recall/(precision + recall))

A confusion matrix, also referred to as an error matrix, is a tool that helps assess and predict the validity of a classification model. Using confusion matrices allows identification of the different errors that occur during model predictions. The most common representation of a confusion matrix is a grid, which is used to determine the accuracy of classification models. This is shown in

Table 3. Model performance metrics.

	Precision	Recall	Precision/Recall	F1 Score	Accuracy
Value	0.932	0.99	0.845	0.78	0.972

, and a comparative chart is shown in Figure 9.

6. Conclusions/Future Work

Many farmers are trying to improve the quality and yield of their crops. At the same time, people are becoming more conscious of the need to protect the environment. So, image and video processing are being used. Models like these will help farmers to achieve not only their desired production but also help their passive goal of protecting the environment. This demonstrates the importance of the proposed method, which will provide much higher accuracy with decreased time and money costs compared to previous models.

This model can also be used as a portable mobile app that is available for farmers to scan their crops, identify weeds and their types, and select the proper management for them.