Three-Dimensional Structure Measurement for Potted Plant Based on Millimeter-Wave Radar

Abstract

Potted plant canopy extraction requires a fast, accurate, stable, and affordable detection system for precise pesticide application. In this study, we propose a new method for extracting three-dimensional canopy information of potted plants using millimeter-wave radar and evaluate the system on plants in static, rotating, and rotating-while-spraying states. The position and rotation speed of the rotating platform are used to compute the rotation–translation matrix between point clouds, enabling the multi-view point clouds to be overlaid on the world coordinate system. Point cloud extraction is performed by applying the Density-Based Spatial Clustering of Applications with Noise algorithm (DBSCAN), while an Alpha-shape algorithm is used for three-dimensional reconstruction of the canopy. Our measurement results for the 3D reconstruction of plants at different growth stages showed that the reconstruction model has higher accuracy under the rotation condition than that under the static condition, with average relative errors of 41.61% and 10.21%, respectively. The significant correlation between the sampling data with and without spray reached 0.03, indicating that the effect of the droplets on radar detection during the spray process can be neglected. This study provides guidance for plant canopy detection using millimeter-wave radar for advanced agricultural informatization and automation.

1. Introduction

Potted plants are becoming more and more popular among consumers because of their aesthetic appeal and long-lasting beauty. Their production has grown in scale in recent years, creating a large demand for labor [1]. One of the most important and laborious tasks is preventing and controlling plant diseases and pests [2]. The current method of applying pesticides to potted plants mainly depends on manual labor [3], which is strenuous and requires managers to have knowledge of plant pathologies [4]. To protect their plants from diseases and pests, most managers tend to spray too much pesticide, leaving excessive residues on the potted plants and even contaminating the environment and soil around them [5].

Precise pesticide application technology is a widely recognized solution to reduce waste and pollution [6]. Using data on plant canopy height, volume, diameter, etc., can increase the efficiency of pesticide use and promote reliable spraying and equipment usage [7]. The technology involved in collecting potted plant canopy information is crucial for accurate spraying [8], and sensors provide essential hardware support for this technology.

The combination of advanced sensing technologies such as computer vision [9,10] and LIDAR (light detection and ranging) [11,12] with conventional agricultural equipment has led to the emergence of a new generation of modern precision spraying machines and devices [13]. The agricultural environment is dynamic and uncertain, which makes it challenging to collect target information. For optical sensors, light and temperature affect the image quality of cameras, while raindrops can absorb or reflect emitted light, reducing the measurement capability of LIDAR [14]. As a stable alternative and complementary solution for perception tasks, millimeter-wave radar uses radio detection and ranging, and has gained attention in recent years [15]. Researchers have examined the use of millimeter-wave radar for various purposes in agriculture, such as guiding robots in smoky mazes [16], identifying subtle human actions [15], generating maps of orchards for automatic control [17], and building systems for predicting grape production [18] and assessing tomato sugar levels [19]. A new technique proposed by Nashashibi et al. calculated the extinction and volume backscattering coefficients of different tree canopies under different physical conditions to improve the detection accuracy of millimeter-wave radar in plant canopies [20]. Their research showed that millimeter-wave radar has strong anti-interference and perception capabilities in complex and volatile environments. However, the application of millimeter-wave radar in precision spraying remains under-reported.

In this paper, we propose a low-cost and fully automatic 3D reconstruction platform for potted plants based on millimeter-wave radar that can effectively and accurately perform 3D reconstruction of potted plants. After collecting the point cloud information, multi-view point clouds are stitched onto the world coordinate system via coordinate transformation. Filtering and the Density-Based Spatial Clustering of Applications with Noise algorithm (DBSCAN) are then used to extract critical point clouds from the data. The Alpha-shape algorithm is applied for three-dimensional reconstruction, allowing the phenotypic parameters to be measured without damaging the plants. Finally, we discuss the application prospects of millimeter-wave radar in precision spraying by comparing the performance of the proposed system under different spraying conditions.

2. Materials and Methods

2.1. Detection System

An experimental test system was constructed to appraise the efficacy of millimeter-wave radar in detecting potted plant canopy, as shown in Figure 1. The system is composed of different parts, including a rotating platform, millimeter-wave radar, potted plants (Epipremnum aureum), and spraying system. An individual plant was centrally positioned on the rotation platform. The main part of the spraying system was an electric-driven sprayer which could achieve a flow rate of 1.2 L/min under a pressure of 0.7 MPa. During the experiments, the plant was rotated by adjusting the rotating platform to 1 r/min, at which the spraying system was able to evenly and continuously spray the pesticide onto the plant. All of the experiments were carried out indoors while strictly controlling the environmental conditions in order to ensure the reliability of the results.

Millimeter-wave radar relies on echo intensity to determine the target’s distance, then generates a point cloud density that varies at different distances [21,22]. To mitigate the effects of distance on data acquisition, the distance between the radar and the potted plants was fixed at a distance of 1.5 m (distance A in Figure 1a). The millimeter-wave radar we employed (DB77M34A6035V2, Xian Dabao Electronic Technology Co., Ltd., Xi’an, China) possessed an extensive detection range of 120° on the horizontal plane and 50° on the vertical plane. Further details regarding the millimeter-wave radar are provided in

Table 1. Millimeter-wave radar parameters.

Parameter	Value
Operational frequency band/GHz	77–81
Refresh rate/Hz	10
Horizontal beam width/°	±60
Vertical beam width/°	±25
Measuring range/m	<9
Range resolution/cm	4
Horizontal angle resolution/°	15
Range accuracy/cm	±2
Horizontal measurement accuracy/°	±1
Vertical measurement accuracy/°	±2

. To ensure that the radar was appropriately configured to detect the target, it was positioned 0.8 m above the ground (distance E in Figure 1a) and rotated 90° counterclockwise to extend its vertical detection range. The point cloud data of the potted plants was collected by using Demo Visualizer developed by Texas Instruments.

2.2. Data Processing

The point cloud data of the potted plants were collected in static, rotating, and rotating-while-spraying (simulating a real working environment) states using millimeter-wave radar, and analysis was performed. However, the point cloud data we acquired unavoidably contained noise, background, and other irrelevant point clouds, resulting in a large amount of inapplicable point cloud data. To mitigate this problem, a preprocessing step to extract the point cloud of the target plant from the raw data was essential in order to reduce the computational cost and improve the reconstruction efficiency. Figure 2 depicts the data processing in three parts: (i) The point cloud data were collected and processed using the Constant False Alarm Rate (CFAR) method to eliminate the noise; (ii) analysis and comparison of the point cloud data of the background and the detection target were undertaken to extract the canopy data; and (iii) Matlab R2017b (MathWorks, Natick, MA, USA) was utilized to reconstruct the canopy of the potted plants while analyzing the canopy information, such as the height, width, and volume of the plants. After this process was complete, the results were compared to the initial measurements in order to determine the detection ability of the millimeter-wave radar.

2.2.1. Point Cloud Data Preprocessing

The proposed potted plant detection system incorporates a rotating platform in conjunction with a millimeter-wave radar to generate multi-view point clouds of potted plants at predetermined angles. Consequently, while the rotating platform drives the potted plants to rotate uniformly, the entire data acquisition system is characterized by a world coordinate system, a millimeter-wave radar coordinate system, and a rotating platform coordinate system. The coordinate system of the rotating platform coincides with the world coordinate system when the platform is stationary.

During the detection process, the millimeter-wave radar utilizes a high-frequency circuit to generate electromagnetic waves of a specific modulation frequency (FMCW) with itself as the origin. The radar sends out electromagnetic waves through an antenna and receives the waves reflected back from the target [23]. By analyzing the parameters of the sent and received electromagnetic waves, the various parameters of the target can be calculated. As a result, the detected target points are expressed in polar coordinates, as illustrated in Figure 3a. Therefore, it is crucial to transform the polar coordinate data into the Cartesian coordinate system before performing point cloud matching and 3D reconstruction. As the millimeter-wave radar was installed after a 90° rotation, it coincides with the world coordinate system following rotational transformation using Equation (1).

$$\left(\begin{array}{c}x\\ y\\ z\end{array}\right)=\left(\begin{array}{ccc}0& 0& R\\ 0& R& 0\\ R& 0& 0\end{array}\right)\left(\begin{array}{c}cos\phi sin\theta \\ cos\phi cos\theta \\ sin\phi \end{array}\right)$$

(1)

In the above equation, R, $\phi$, and $\theta$ respectively represent the distance, elevation angle, and roll angle of point P in the polar coordinate system returned from the radar, while x, y, and z represent the Cartesian coordinate system of point P in the world coordinate system.

When the rotating platform is stationary, the platform coordinate system coincides with the world coordinate system. In this case, the potted plant data acquired by the radar do not require registration and can be directly used for three-dimensional reconstruction. However, when the potted plant rotates with the rotating platform, different sampling frames need to be correlated and matched to ensure that the point clouds of each sampling frame can be superimposed. Because the z-axis of the rotating platform coordinate system coincides with the z-axis of the world coordinate system, only the data on the X–Y plane need to be calculated, as shown in Figure 3b. A fixed angle difference $\beta$ exists between the x and y values of each frame and the previous frame, which is determined by the speed of motion of the rotating platform and the sampling period of the radar. In order to match the point cloud data of the previous frame to the current moment each point cloud was calculated at the initial angle and canopy radius, starting from the second frame of the data, via the following equation:

$$\left\{\begin{array}{c}\alpha =arctan\frac{x_1}{x_1-A}\\ r=\sqrt{x_1^2+{(y_1-A)}^2}\end{array}\right.$$

(2)

where A represents the distance from the radar to the middle of the rotating platform and ($x_1$, $y_1$) represents its initial coordinate when matching the target point cloud.

Then, the point cloud of the previous frame can be matched to the current frame and remapped continuously; for example, in the i-th sampling frame, the equation is expressed as follows.

$$\left\{\begin{array}{c}{x}_i=r·sin(\alpha +(i-1)\beta )\\ y_i=r·cos(\alpha +(i-1)\beta )\end{array}\right.$$

(3)

After calculating all the point clouds of the current frame and matching all the frames, the complete sampling point cloud information of the current moment can be obtained.

2.2.2. Cluster Analysis

The canopy information of the potted plants can be obtained by clustering the point cloud information. Compared with the stationary state, the potted plants in the rotating state can obtain point cloud sampling information from multiple angles, thereby increasing the amount of point cloud data. After eliminating the environmental point cloud noise from the roof or the ground, it is necessary to define the region of interest (ROI) according to the relative position of the potted plants and the millimeter-wave radar in order to extract the target point cloud information more accurately and effectively.

To achieve this, we utilized a statistical filtering algorithm to effectively remove any outlier noise that might remain after establishing the ROI. Statistical filters are efficient algorithms that establish the nearest average distance to each adjacent point based on the Euclidean distance formula, then divide the selection area according to the mean and standard deviation of the distance to eliminate invalid points. After successfully eliminating outlier noise using the statistical filtering algorithm, the next step involved utilizing the Density-Based Spatial Clustering of Applications with Noise algorithm (DBSCAN) to remove any remaining rotated platform point clouds and extract the potted plant canopy features. The DBSCAN algorithm is an ideal choice for this task due to the sparse nature of potted plant canopy point cloud data [24]. As a density-based clustering algorithm, DBSCAN identifies closely connected high-density intervals within the datasets as clusters and separates different clusters based on low-density intervals [25]. This algorithm can efficiently extract canopy data from datasets of any size and shape without the need to preset the number of clusters. Subsequently, the remaining clusters were reconstructed into a 3D structure utilizing the Alpha-shape algorithm, which is an effective means of generating a 3D structure from point cloud data. The resulting 3D structure parameters such as height, diameter, and volume were meticulously compared to manual measurements to ensure optimal precision and accuracy. The interplay between the various algorithms employed throughout this process affects the outcome; therefore, the parameters of each algorithm are crucial to achieving the best results.

Figure 4 shows the point cloud data processing approach for the potted plants under static, rotating, and rotating-while-spraying conditions. The noise threshold, neighborhood radius epsilon (Eps), and the density threshold were set to 170, 0.05, and 9, respectively, for better processing results. Because the coordinate system is relative to the millimeter-wave radar measurement, the result is a negative z value. The estimated height of the canopy is the sum of the absolute values of the maximum detection distances in the vertical direction (the z-axis).

2.2.3. Calculation of Potted Plant Canopy Volume

In this study, the volume of the detection target was calculated by reconstructing the three-dimensional model of the potted plants using the Alpha-shape algorithm. The basic idea of this algorithm is to roll a sphere or circle with radius Alpha along the point set to form a contour boundary line. The Alpha-shape algorithm can reduce the overestimated areas compared with the convex hull algorithm, and as such has better boundary estimation ability [26]. The Alpha value in the Alpha-shape algorithm is an important parameter that determines the fineness of the boundary [27], and should be adjusted to ensure that it is reasonable according to the current point cloud density. In our experiments, we used alpha values of 0.1, 0.3, 0.6, and 1 to evaluate the volume of the potted plants.

Because the shape characteristics of the canopy of Epipremnum aureum are similar to a cylinder, the actual volume of the canopy used for comparing the reconstruction effect needs to be calculated based on the diameter and height measured manually ($V=\Omega D^{2}H/4$). In order to reduce the error, the average values of the north–south diameter $D_{NS}$ and east–west diameter $D_{EW}$ were adopted as the diameter of the canopy, as shown in Figure 1a, and the average value of the measured maximum height H of the potted tree crown was adopted as the height of the canopy.

3. Results and Discussion

3.1. Canopy Identification Results for Potted Plants

Using a millimeter-wave radar, a cluster of ten potted plants of varying growth stages was gathered under the static, rotating, and rotating-while-spraying conditions. The errors of the millimeter-wave radar relative to the manual measurements under different plant conditions are shown in Figure 5. The relative error was reduced when the plants were measured under rotation due to the increased amount of sampling point cloud data compared to the static condition. The significant correlations between the number of point clouds obtained under different conditions were calculated, and the results are shown in Figure 6. The point density of the plants under the static condition had poor correlation with the rotating and the rotating-while-spraying conditions (p = 0.42 and 0.53), while the number of datapoints recorded under the rotation condition exhibited a strong correlation with the number obtained under the rotating-while-spraying condition (p = 0.03). These results show that the millimeter-wave radar had strong anti-interference performance against fog droplets, making it suitable for dynamic and uncertain agricultural environments.

In order to assess the precision of the extracted canopy information, a comparative analysis was conducted between the height and diameter measurements of the potted plants using both the millimeter-wave radar and manual measurement methods. The distance distributions of the corresponding points between the canopy parameters were calculated to evaluate the similarity based on the Root Mean Square Error (RMSE). The RMSE is a quantifiable indicator of the accuracy of millimeter-wave radar measurements, with a lower RMSE corresponding to a higher level of accuracy. The statistical results for the $R^2$ value and RMSE value for height and diameter under different states are shown in

Table 2. Summary of the $R^2$ and RMSE results in different conditions.

State	${\mathit{R}}_{\mathit{Height}}^2$	${\mathbf{RMSE}}_{\mathbf{Height}}$	${\mathit{R}}_{\mathit{Diameter}}^2$	${\mathbf{RMSE}}_{\mathbf{Diameter}}$
Static	0.48	551.35	0.838	214.25
Rotating	0.994	115.46	0.989	61.68
Rotating and Spring	0.995	127.78	0.998	60.44

. The RMSE values for the height and diameter measurements of the potted plants under the static condition were found to be significantly higher, at 551.35 mm and 214.25 mm, respectively, compared to the measurements obtained under the rotating (115.46 mm and 61.68 mm) and rotating-while-spraying (127.78 mm and 60.44 mm) conditions. These results indicate that the measurement conditions significantly influenced the accuracy of the millimeter-wave radar measurements, with the rotating and rotating-while-spraying conditions yielding more accurate results.

Fitting equations for the measurement results of the potted plants in the three different conditions were constructed, as shown in Figure 7. Compared with manual measurement, the average relative errors for the height and diameter of the potted plants as measured by millimeter-wave radar were 45.15% and 38.07%, respectively, for an average error of 41.61%. On the contrary, the relative errors of the height and diameter in the rotating state were significantly lower at 10.25% and 10.18%, respectively, for an average error of 10.21%. These results indicate that the millimeter-wave radar exhibits superior detection performance for dynamic targets compared to static ones, which can be attributed to two factors. First, the resolution of the millimeter-wave radar was low, resulting in the loss of data on the target edge, particularly when the potted plants were stationary. Second, millimeter-wave radar detection relies on the Doppler effect principle, endowing it with a high Doppler frequency and making for outstanding detection and identification capability for low-vibration targets.

In the rotating-while-spraying state, the average relative errors were 11.01% and 10.4%, for an average error of 10.7%. This result is close to the 10.21% obtained under the rotating condition, indicating that the detection process of the millimeter-wave radar was not significantly affected by the spraying operation.

3.2. Three-Dimensional Reconstruction and Volume Calculation of Potted Plant Canopy

According to the principle of the Alpha-shape algorithm, the reconstructed 3D volume is different with the different values of the Alpha parameter. The potted plant canopy parameters (height and diameter) were reconstructed in three dimensions with Alpha values of 0.1, 0.3, 0.6, and 1, and the linear correlation with the manual measurement values was computed. The results are shown in

Table 3. Coefficient of determination $R^2$ at different Alpha values.

Alpha	${\mathit{R}}_{\mathit{Height}}^2$	${\mathit{R}}_{\mathit{Diameter}}^2$
0.1	0.48	0.84
0.3	0.75	0.92
0.6	0.99	0.98
1	0.89	0.91

. It can be seen that when the Alpha value was 0.6, the $R^2$ was the largest ($R_{Height}^2$ = 0.99, $R_{Diameter}^2$ = 0.98), indicating that the 3D reconstruction result was te closest to the actual value. Taking the No.1 potted plant as an example the 3D reconstruction results of the potted plant in the static, rotating, and rotating-while-spraying conditions when Alpha = 0.6 are shown in Figure 8. In the static condition, the reconstructed structure differs greatly from the real structure due to the insufficient number of point clouds detected by the millimeter-wave radar. Under the rotating and rotating-while-spraying conditions, on the other hand, the reconstructed potted plant structure closely resembles the real potted plant.

A standardized procedure was applied to investigate the volumes of ten potted plants (Epipremnum aureum) at different stages of growth, revealing that the movement state of the potted plant had a significant impact on the reconstructed canopy volume. The volume of the reconstructed model was considerably smaller than that of the actual potted plant when the latter was stationary, as illustrated in Figure 9. Conversely, the reconstructed structure matched well with the real canopy under the rotating and rotating-while-spraying conditions, with an RMSE of about 0.00813 m³. When the potted plant was static, the millimeter-wave radar only captured the point cloud data of the potted plant from one perspective, resulting in less comprehensive point cloud data in the static state than in the rotating state. Therefore, the volume of the potted plants measured by the millimeter-wave radar tended to be smaller when the plant was in a static position compared to when it was rotating.

It should be noted that the canopy of potted plants consists of complex overlapping branches and leaves, which poses a challenge for manual measurement. The need to eliminate the gaps within the canopy leads to a larger measured volume of than the 3D reconstructed volume. Although the millimeter-wave radar employed in this study had low resolution and accuracy, hindering direct acquisition of the exact canopy volume, the collected point cloud boundary data effectively reproduced the uneven surface of the canopy. Therefore, the canopy volume obtained in this study is more accurate and closer to the actual canopy volume than the canopy volume derived from traditional geometric measurement methods.

In [28], Wang achieved a detection accuracy of 99.18% based on LIDAR for the canopy volume of peach trees, while in [29] Zhu used computer vision to achieve a detection accuracy of 1.38 cm for tomato plant canopy. However, refined three-dimensional reconstruction requires better equipment, increasing the overall cost. For potted plants with dense leaf canopies, processing the point clouds obtained by millimeter-wave radar into tetrahedral networks can quickly obtain the position and volume of the canopy, which can then be used to guide the spray area and dosage. At the same time, thanks to the anti-interference characteristics of millimeter-wave radars it is possible to quickly and accurately reconstruct potted plant canopies even with droplet interference during spraying operations.

4. Conclusions

In this study, we have presented an approach for detecting potted plant canopies based on millimeter-wave radar. The proposed method utilizes a rotating platform to collect point cloud data of potted plants. We tested the proposed system in three different conditions: static, rotating, and rotating-while-spraying. The point cloud data were then processed by multi-view collection, preprocessing, registration, and smoothing to achieve rapid 3D reconstruction. Structural measurement of the canopy was then performed based on the reconstructed three-dimensional plant model.

The canopy reconstruction effect was significantly better under rotation than under the static condition, indicating that the resolution of the millimeter-wave radar was insufficient and could not accurately calculate the canopy information based on single-sided data. Moreover, the number of point clouds collected by millimeter-wave radar under the rotating and rotating-while-spraying states had a high correlation with the canopy reconstruction effect, indicating that droplet interference during the spraying operation had no significant impact on the detection of canopy information by the millimeter-wave radar. This study provides technical support and experimental verification for the application of millimeter-wave radar for detecting the canopy information of potted plants, and could additionally be applied to other plants such as fruit trees.