Study on Real-Time Detection of Lightweight Tomato Plant Height Under Improved YOLOv5 and Visual Features

Abstract

Tomato cultivation is relatively dense, and the main stem is easily submerged in a background environment with small color difference. The semi-enclosed planting space and fast growth cycle are both limitations that cannot be ignored in detection technology. The accuracy and real-time performance of plant height detection are of great practical significance. To this end, we are committed to improving YOLOv5 and proposing a lightweight real-time detection method for plant height by combining visual features of tomato main stems. Here, we improved the backbone, neck, head, and activation functions of YOLOv5, using CSP dark net53-s as the backbone structure and introducing a focus structure to reduce the number of GE modules. We replaced all CSP2_X structures in neck and head with GE modules, embedded interactive multi-head attention, and replaced YOLOv5’s framework function and attention activation function. We defined visual features such as the color of the main stem of tomato plants in the preprocessed image; input improved YOLOv5; and completed plant height detection through effective feature map fusion, main stem framing, and scale conversion. The experimental results show that the linear deviation between the plant height detection value and the actual value of the proposed method is always less than 3 cm, and the detection FPS can reach up to 67 frames per second, with superior timeliness, which can effectively achieve lightweight real-time detection.

1. Introduction

With the continuous progress of agricultural science and technology and the increasing demand for precision agriculture, the real-time monitoring and evaluation management of crop growth parameters has become particularly important [1]. As one of the key indicators to measure crop growth, plant height can reflect plant growth rate, health status, adaptability to the external environment, and possible yield trend [2]. Therefore, how to obtain plant height accurately and quickly has become the exploration direction of agriculture, forestry, ecology, and many other fields. By obtaining crop plant height, farmers and researchers can not only screen varieties with excellent traits, improve the genetic material of crops, optimize the growing environment according to the growth conditions of crops, and provide a basis for ecological protection and environmental governance, but also predict the yield trend of crops and provide support for resource allocation and planting management decisions in agricultural production [3,4].

At present, there are two main ways to obtain plant height: direct and indirect. With the continuous progress of advanced technologies such as computer vision, the means of obtaining plant height are constantly updated and improved, making indirect nondestructive testing technology gradually attract extensive attention from scholars at home and abroad, and many methods have been proposed in this regard, but there are still some problems [5,6]. For example, in the lettuce plant height detection method proposed by He Xingyao et al. [7], only the attention mechanism of YOLOv5 was improved, and the interference of farmland environment was not suppressed. Moreover, errors would be generated during image processing by UAV oblique photography. The UAV method proposed by Wu Tingting et al. [8] for measuring wheat plant height ensured the accuracy of acquiring plant height phenotype by UAV, but the remote sensing operation was time-consuming and inefficient. In the UAV RGB remote sensing estimation method of sugarcane plant height proposed by Liang Yongjian et al. [9], the RGB model relies too much on the reflection information of the red, green, and blue wave segments, and the illumination intensity and angle, as well as the overdensity of the plant, will affect the estimation accuracy. In the prediction model constructed based on near-infrared, Raman, and fluorescence spectra, Johannes et al. [10] found that different excitation light sources of fluorescence spectra and changes in farmland environment would all lead to spectral differences and fluctuations, making it impossible to obtain accurate wheat plant height. In the maize plant height estimation method based on the fusion of Sentinel-1 dual-polarization SAR data, the backscattering coefficient, and depolarization parameters studied by Wang et al. [11], although the inversion accuracy in the later period of crop growth was improved, the Sentinel-1 satellite had a long revisit period and could not obtain the continuous growth data of maize in time [12]. And the surface factors would have interfered with the SAR signal, resulting in estimation errors.

For tomato, as a widely grown vegetable crop, the growth rate of change is fast, there is not enough operation time, and it is usually planted in a greenhouse, which means that the operating space is extremely limited and it is not a suitable environment for the operation of drones [13]. At the same time, tomato planting is relatively dense, and the main stems of different plants can easily be confused with each other, but also with other environmental backgrounds such as leaves with small color difference, which increases the difficulty of plant height detection. Therefore, based on the backbone network architecture of lightweight YOLOv5, a lightweight real-time plant height detection method was designed by introducing the visual features of tomato appearance [14]. The lightweight method improves various aspects of YOLOv5, meaning that YOLOv5 can meet the dual needs of plant height detection: speed and accuracy. Defining the visual features of tomato main stems, such as color, texture, and shape, can more accurately locate the detection target from a complex background, retain useful information about the main stem, and thus ensure the detection accuracy of plant height and increase real-time performance.

2. Improvement in YOLOv5 for Real-Time Detection of Plant Height of Lightweight Tomato

In terms of its applicability to the specific task of the real-time detection of tomato plant height, compared with the YOLOv8 and YOLOv11 versions, YOLOv5 has a structure that is more easily modified and optimized and can better adapt to the task. YOLOv5 is more lightweight in terms of computational resource requirements and is more suitable for running in the small model designed in this paper. YOLOv5 is improved in terms of backbone, neck, head, and so on in order to simplify the model scale and the number of parameters and strengthen the model’s target recognition ability against a complex background, aiming to achieve the light weight and real-time detection requirements and handle the density of tomato plant height detection targets.

The improved YOLOv5 model is shown in Figure 1, which is the main resource used to realize the real-time detection of tomato plant height and weight.

Specific improvement methods in various aspects of the model are described as follows:

(1)

Backbone: The backbone of YOLOv5 is replaced with CSP dark net53-s [15], and GE modules are reduced to achieve a lightweight structure and simplified computation. A ghost module [16] and Mobile net v2 [17] are combined to establish the ghost bottleneck structure of the GE module to prevent information loss due to scale compression by converting information between small dimensions and large dimensions. In order to compensate for the precision loss of the structure and make the information about the feature map more complete and rich, the GE module channel is extended and the focus structure [18] is introduced to slice the input image.

(2)

Neck and head: First, all the CSP2_X structures in the neck and head are replaced with GE modules, and all standard convolution is replaced with deep separable convolution [19], so as to avoid model parameter redundancy and reduce the calculation amount. Then, interactive multi-head attention is embedded into the neck [20], and after obtaining spatial attention and channel attention simultaneously, feature information is selected based on two dimensions.

(3)

Activation function: In order to deepen the information in the neural network, the framework function of YOLOv5 is replaced by the swish function [21]. The embedded attention activation function is replaced by the H-sigmoid function [22] to expand the detection accuracy.

3. Real-Time Detection of Tomato Plant Height and Weight

The improved YOLOv5 model plays a core role in real-time detection and can adapt to the real-time detection task of lightweight tomato plant height. Therefore, it is used as the implementation body of the real-time detection of lightweight tomato plant height. The main stem color, texture, shape, and other visual features captured from tomato images are input into the improved YOLOv5 model. It can capture higher-quality visual features and better adapt to the complex and dense background environment, without the influence of lighting conditions, occlusion, shooting angle, and other factors, identifying the main stem of tomato plants [23], and finally, through layer-by-layer reasoning, obtaining the plant height detection results.

The real-time detection process of lightweight tomato plant height based on improved YOLOv5 is described as follows:

(1)

Delineation of visual characteristics of main stems of tomato plants. Regarding the visual characteristics of the main stem, the color, texture, and shape are selected to avoid deviation due to single features.

(a) Color is captured by color moments that do not need to quantify color features [24], that is, the formula for calculating the dispersion σ_i and skew s_i of color distribution is as follows:

$$\left\{\begin{array}{l}{\sigma }_i={\displaystyle \frac{N^2}{{\left[{\displaystyle \sum _{j=1}^N{\left(\frac{N{p}_{ij}-{\displaystyle \sum _{j=1}^N{p}_{ij}}}{N}\right)}^2}\right]}^2}}\\ s_i={\displaystyle \frac{N^3}{{\left[{\displaystyle \sum _{j=1}^N{\left(\frac{N{p}_{ij}-{\displaystyle \sum _{j=1}^N{p}_{ij}}}{N}\right)}^3}\right]}^3}}\end{array}\right.$$

(1)

In the above formula, P_ij stands for the i-th color value component of pixel j in N pixels.

(b) Texture is captured by a gray co-incidence matrix based on the gray level spatial correlation matrix [25]. In order to reduce the amount of computation and remove the adjacent interval and direction information about image gray level in the matrix, the texture of the main stem of a tomato plant is obtained through the following L gray level optimization formula:

$$R\left(g,h\right)={\left[\begin{array}{cccc}{r}_{00}& r_{01}& \dots & r_{0\left(L-1\right)}\\ r_{10}& r_{11}& \dots & r_{1\left(L-1\right)}\\ \dots & \dots & \dots & \dots \\ r_{\left(L-1\right)0}& r_{\left(L-1\right)1}& \dots & r_{\left(L-1\right)\left(L-1\right)}\end{array}\right]}_{L\times L}$$

(2)

In the above formula, (g,h) stands for gray level, g,h = 0, …, L – 1, and matrix element rgh stands for counter, which is used to count all the pixels in the image that meet the following equations:

$$\left\{\begin{array}{l}G\left(x,y\right)=g\\ G\left(x+d_x,y+d_y\right)=h\end{array}\right.$$

(3)

In the above formula, G(x,y) stands for the gray level of pixel (x,y), and d_x and d_y stand for the offset in different directions.

(c) The shape is mainly captured through curve fitting [26]. First, according to the vertical characteristics of the main stem, the key points are set equidistant in the vertical direction along the contour curve. The equal spacing values D_left and D_right of the left and right contours of the stems are solved using the following two formulas:

$$\left\{\begin{array}{l}{D}_{left}={\displaystyle \frac{l}{n_{left}}}\\ D_{right}={\displaystyle \frac{l}{n_{right}}}\end{array}\right.$$

(4)

In the above formula, l stands for the length of the major axis of the elliptic model used to fit the main stem, and n_left and n_right stand for the key points on the left and right outlines.

The following conics are used to fit the key points of the left and right contours, and their real-time properties are expanded to obtain the fitting curve parameter, that is, the visual feature vector (α₁,α₂,α₃,β₁,β₂,β₃) of the shape, as follows:

$$\left\{\begin{array}{l}{Y}_1={\alpha }_1{X}^2+{\alpha }_{2}X+{\alpha }_3{D}_{left}\\ Y_2={\beta }_1{X}^2+{\beta }_{2}X+{\beta }_3{D}_{right}\end{array}\right.$$

(5)

(2)

Three types of visual features are input into the improved YOLOv5 model, and through the backbone, neck, and head parts and step-by-step operation and reasoning, effective feature maps of different specifications of receptor fields are obtained. The main stems of tomato plants are selected from the background by using the feature pyramid network [27] and three effective feature maps.

(3)

According to the obtained ratio value B of actual plant height H₀ and detected plant height $H_0^{\prime }$ the following ratio conversion is carried out to solve the actual plant height H_t of tomato and complete the test:

$$H_t={\displaystyle \frac{H_0}{H_0^{\prime }}}\times H_t^{\prime }=B\times H_t^{\prime }$$

(6)

In the above formula, $H_t^{\prime }$ stands for frame. The height of the outer frame of the main stem is chosen.

4. Experiments

In the demonstration base of the Agricultural Extension Center of Zhongshan City, Guangdong Province, multiple images of tomato varieties planted in solar greenhouses were collected. The field environment of image collection and plant height detection is shown in Figure 2.

Images of tomato plants grown in solar greenhouses were collected using a Parrot Sequoia multispectral camera, which contains an RGB sensor. The RGB sensor integrated the collected images with the plant height detection data to prepare the data set for the experiment. The data set included 10⁵ images. Labels were divided based on the actual height of the tomato main stem. Labels were classified according to a height of 0.1 m. The highest tomato main stem height was 2.5 m, and the lowest was 0.3 m, and the labels were divided into 22. The image collection and plant height detection data were integrated to prepare the experimental data set. The data in the data set were cleaned and preprocessed. The images were checked to determine whether they had overexposure. All images were adjusted to a resolution of 1920 × 1080 pixels for unified processing. Edge detection methods were used to extract the shape information about the plant height in the images, and the data were augmented. The data set, after augmented processing, was divided into a 70% training set and a 30% test set. A representative image in the training set is shown in Figure 3.

After the data set was imported into the image processing software (OpenCV 4.5, Intel, Santa Clara, CA, USA) for filtering, enhancement, normalization, and other preprocessing steps, 1000 pieces were randomly selected as the training set and the rest as the test set by the random sampling program. The improved YOLOv5 model was trained using the processed tomato image training set. The trained model was applied to the real-time detection task of the test set, run by Pytorch 2.1.1, Python 3.10.0 development framework, and trained and tested by an intel core TM i9 processor and NVIDIA 3080Ti graphics card (12 GB). The tomato plant height in the image set was measured quickly and accurately.

In the training stage, the initial training parameters of the stochastic gradient descent optimizer and YOLOv5 model for training were set, as shown in

Table 1. Hyperparameter setting based on model training.

Module	Argument	Numerical Value
Stochastic gradient descent optimizer	Learning rate	0.02
	Momentum coefficient	0.965
	Weight attenuation	0.0003
YOLOv5 model	Batch size	48
	Epoch number	200

.

Bayesian optimization was used to tune the hyperparameters. Fifty values were selected for each hyperparameter for the grid search. Combinations were randomly selected in the hyperparameter space for attempts, and resources in the space were reasonably allocated to select the optimal hyperparameter combination. After 500 iterations, the best hyperparameter combination was obtained: the learning rate was 0.06, the momentum coefficient was 0.875, the weight decay was 0.00027, the batch size was 45, and the number of epochs was 210.

4.1. Ablation Experiment of Improved YOLOv5 Algorithm

From the perspective of the parameter (pixel granularity) amount, calculation amount, and pixel granularity accuracy of the YOLOv5 algorithm [28], an ablation study on YOLOv5, YOLOv5 with improved backbone (i.e., B-YOLOv5), YOLOv5 with improved neck (i.e., N-YOLOv5), YOLOv5 with improved head (i.e., H-YOLOv5), and three types of YOLOv5 (i.e., BN-YOLOv5, NH-YOLOv5, BH-YOLOv5) obtained by implementing the three improvement strategies in different orders, as well as YOLOv5 obtained by implementing all three improvement strategies (i.e., the improved YOLOv5 model shown in Figure 1), was conducted to investigate the performance of the improved YOLOv5 model. The experimental results are shown in Figure 4. Among them, the standard accuracy was 0.8, the recall rate was 0.7, the value of mAP 0.5 was 0.65, and the value of mAP 0.95 was 0.5. The pixel granularity accuracy was divided into @0.5 and @0.4:1, which, respectively, represent the average pixel granularity with the IOU (intersection ratio) threshold of 0.5 and within the range of 0.4~1 of 0.06 steps.

As shown in Figure 4, the parameter number and scale of different YOLOv5 improvement methods show that the parameter number of the improved YOLOv5 algorithm after the proposed method was reduced by 6.61 M, the calculation amount of the algorithm was simplified to 3.84 MB, and the granularity accuracy of @0.5 pixel was improved by 0.101 and @0.4. The granularity accuracy of 1 pixel was improved by 0.172; this is because when the main stem color, texture, shape, and other visual features captured from tomato images were input into the improved YOLOv5 model, the model could capture visual features of higher quality, thus giving YOLOv5 better detection performance.

4.2. Deviation and Speed Test of Plant Height Detection

For tomato crops in the study area, the UAV measurement method, multi-spectral fusion measurement method, and designed detection method were used to obtain the plant height [29,30]. By comparing the linear deviation of the measured plant height and the level evaluation of FPS (frames per second), the real-time detection performance of the proposed method was analyzed [31]. We selected any planting ditch in the study area, and the plant height detection of tomatoes in the single ditch is shown in Figure 5.

In Figure 5a, compared with other methods, the linear deviation between the measured plant height and the actual value of the method proposed in this paper was the lowest, which was always less than 3 cm. However, due to the limitation of aerial photography height and insufficient light, the deviation of tomato plant height under 1 m was the largest. In Figure 5b, the FPS timing trends of different methods show that the detection speed of the proposed method was significantly faster, up to about 67 frames per second, while the FPS of the UAV measurement method was no more than 55 frames per second [32], and the FPS of the multi-spectral fusion measurement method was no more than 45 frames per second, showing poor real-time performance.

In order to further verify the effectiveness of the proposed method, the proposed method was compared with YOLOv8 and YOLOv9, and the results of real-time tomato plant height detection are shown in Figure 6.

In Figure 6a, the linear deviation between the measured plant height and the actual value of the proposed method was the lowest, which was always less than 2 cm. This is because the improved YOLOv5 model could perform real-time lightweight calculation on the data, and the resulting deviation was low. However, the results of the YOLOv8, YOLOv9, and YOLOv11 methods were biased due to excessive calculation amounts. In Figure 6b, the detection speed of the proposed method was about 69 frames per second, which was much higher than that of the other three methods, while the FPS of the YOLOv11 method did not exceed 50 frames per second, the FPS of the YOLOv8 method did not exceed 45 frames per second, and that of the multi-spectral fusion measurement method did not exceed 42 frames per second. These existing methods’ real-time performance is poor, indicating the accuracy and practicability of the proposed method.

5. Conclusions

As an important crop, the accurate monitoring of tomato’s growth state is very important for optimizing plant management and improving yield and quality [33]. Plant height is one of the key indicators that reflect the growth of tomato, and it is of great significance for agricultural production to accurately measure it in real time [34]. With the development of modern agriculture, the automatic plant height detection method which does not rely on manual measurement has become the mainstream [35]. However, if the advanced YOLOv5 real-time target detection algorithm is directly applied to plant height detection, there will be problems such as high complexity and large resource requirements [36]. Therefore, with the support of the lightweight and computationally efficient YOLOv5 structure; the simplification of the calculation; the rich information provided by the main stem’s color, texture, shape, and other visual features based on tomato’s multiple visual features [37]; and the effective suppression of the negative interference brought by unideal collection methods and lighting conditions in actual environments, most of the measured values of tomato plant height are basically consistent with the actual height of the main stem [38] and have good detection efficiency. However, under different lighting conditions such as cloudy days and strong light, the visibility and feature recognition of tomato plants in the image will be affected by the model, and the detection of plant height will be affected. In the future, optimizing tomato planting management can be further explored to improve the yield and quality of tomatoes.