A Canopy Information Measurement Method for Modern Standardized Apple Orchards Based on UAV Multimodal Information

To make canopy information measurements in modern standardized apple orchards, a method for canopy information measurements based on unmanned aerial vehicle (UAV) multimodal information is proposed. Using a modern standardized apple orchard as the study object, a visual imaging system on a quadrotor UAV was used to collect canopy images in the apple orchard, and three-dimensional (3D) point-cloud models and vegetation index images of the orchard were generated with Pix4Dmapper software. A row and column detection method based on grayscale projection in orchard index images (RCGP) is proposed. Morphological information measurements of fruit tree canopies based on 3D point-cloud models are established, and a yield prediction model for fruit trees based on the UAV multimodal information is derived. The results are as follows: (1) When the ground sampling distance (GSD) was 2.13–6.69 cm/px, the accuracy of row detection in the orchard using the RCGP method was 100.00%. (2) With RCGP, the average accuracy of column detection based on grayscale images of the normalized green (NG) index was 98.71–100.00%. The hand-measured values of H, SXOY, and V of the fruit tree canopy were compared with those obtained with the UAV. The results showed that the coefficient of determination R2 was the most significant, which was 0.94, 0.94, and 0.91, respectively, and the relative average deviation (RADavg) was minimal, which was 1.72%, 4.33%, and 7.90%, respectively, when the GSD was 2.13 cm/px. Yield prediction was modeled by the back-propagation artificial neural network prediction model using the color and textural characteristic values of fruit tree vegetation indices and the morphological characteristic values of point-cloud models. The R2 value between the predicted yield values and the measured values was 0.83–0.88, and the RAD value was 8.05–9.76%. These results show that the UAV-based canopy information measurement method in apple orchards proposed in this study can be applied to the remote evaluation of canopy 3D morphological information and can yield information about modern standardized orchards, thereby improving the level of orchard informatization. This method is thus valuable for the production management of modern standardized orchards.


Introduction
The canopy is the first part of the fruit tree exposed to light and the external environment and is the main place for photosynthesis and respiration. The shape, structure, volume, leaf area index, vegetation index, and other information of the fruit tree canopy not only reflect the growth status and yield potential of fruit trees [1] but also are important for decision-making about orchard citrus orchards and established a diagnostic model of Huanglongbing (HLB) disease (citrus greening) in citrus orchards, which allowed them to classify fruit trees by HLB disease status. Through a UAV visual system, Park et al. [22] collected high-resolution thermal images of a peach orchard and established a diagnostic model for fruit tree water stress to guide the orchard irrigation management. As a UAV plant protection sprayer can improve the accuracy of target spraying and reduce the application amount, Cheng et al. [5] proposed an automatic segmentation method for canopy images in orchards and established an accurate spraying method based on canopy area. Meanwhile, UAV measurement technology has also been applied in a large body of research in field crops or plants, such as on cotton yield measurement [19], corn height measurement [23], barley biomass measurement [15], wheat height measurement [24], sorghum growth status measurement [25], sunflower yield measurement [26], field weed detection [27], and rapeseed yield measurement [20]. UAV measurement technology is highly efficient and nondestructive and is suitable for large-scale measurement of information about orchards and field crops. Compared with the ground measurement method [28], this technology is more suitable for applications in large, modern precision agricultural production management.
As modern standardized orchards must accommodate agricultural machinery and equipment, they need a large enough row spacing and plant spacing. A typical characteristic of modern standardized orchards is neat rows and columns, yet the presence of a few small trees or dead trees between fruit trees could affect the benefit of canopy segmentation. In this study, taking an apple orchard as the study object, a UAV visual imaging system was used to generate 3D point-cloud models and vegetation indices of the apple orchard. Automatic row and column detection methods were established based on vegetation index images, thereby achieving rapid segmentation. Based on the fruit tree point-cloud models, measurement methods for height, projection area, and projection volume, as well as yield prediction models, were established to achieve a remote evaluation of orchard canopy information. This paper provides a theoretical basis and technical reference for canopy information measurement in modern standardized orchards and will be helpful in orchard management.

UAV Measurement System for Canopy Information of Orchards
The UAV measurement system for modern standardized apple orchards mainly consisted of a Parrot Bluegrass UAV, Parrot Sequoia multispectral cameras, Pix4Dcapture flight control software, Pix4Dmapper postprocessing software, and a graphics workstation. The main technical parameters of the Parrot Bluegrass UAV were as follows: maximal take-off mass (MTOM) is 1.85 kg, battery life of 25 min, Parrot Skycontroller 2 with a control range of 2 km, and a vertical camera equipped with a built-in global positioning system (GPS), global navigation satellite system (GLONASS), altimeter, ultrasonic sensor, and optical flow vertical camera. The main technical parameters of the Parrot Sequoia multispectral camera are as follows: an RGB camera with the resolution of 16 megapixels (4608 × 3456 px); four global-shutter spectral-channel single-band cameras, including green (GRE, 550 ± 40 nm), red (RED, 660 ± 40 nm), red-edge (REG, 735 ± 10 nm), and near-infrared (NIR, 790 ± 40 nm), each with the resolution of 1.2 megapixels (1280 × 960 px) and the frame rate of 1 frame/s, it has four environmental light sensors, these sensors have the same band-pass filter as four single-band cameras, and pictures could be calibrated automatically according to the light conditions. The main functions of the Pix4Dcapture flight control software were as follows: UAV flight map selection, flight path planning, flight altitude, flight speed, horizontal coverage, vertical coverage, and other parameter settings, along with the display function of real-time status information parameters, such as the UAV battery, Parrot Skycontroller battery, memory, GPS, and operation processes. The main functions of the Pix4Dmapper postprocessing software were generating 3D point-cloud models, orthographic images, and DSM and vegetation index images based on the images collected by the UAV. The main technical parameters of the graphics workstation (HP Zhan 99 by Hewlett-Packard Corporation, Beijing, China) were as follows: a 6-core 12-thread Intel (R) Xeon (R) E-2176 M CPU @ 2.70 GHz, 32 GB of Sensors 2020, 20, 2985 4 of 21 RAM, Intel (R) UHD Graphics P630 GPU and NVIDIA Quadro P600 4G graphics card. The operating system was Windows 10 Professional Workstation Edition, 64 bit. The software for calculating the 3D morphological information of the fruit tree canopy was Matlab 2017a (MathWorks Corporation, Natick, MA, USA).
The workflow of the UAV measurement system for canopy information in a modern standardized apple orchard is as follows: The first step was image acquisition. According to the selected orchard measurement area, Pix4Dcapture created a flight (including flight map boundary selection, flight path planning, flight altitude, flight speed, horizontal coverage, vertical coverage, and other parameter settings), the Parrot Bluegrass performed this flight, and the Parrot Sequoia multispectral camera collected multispectral images at each planned waypoint. The second step was the point-cloud output. Pix4Dmapper was used to read images collected by the flights, and according to the GPS information (longitude, latitude, altitude) of each waypoint, Pix4Dmapper carried out processing procedures, such as computing keypoints, computing matches, calibration, matching, and point-cloud densification, and finally output the PLY file of the point-cloud model of the orchard. The third step was the vegetation index output. According to GRE, RED, REG, and NIR orchard canopy orthophotos, Pix4Dmapper was used to generate various vegetation index images (such as normalized difference vegetation index (NDVI), normalized green (NG), green chlorophyll index (CIG), and renormalized DVI (RDVI)), and the orchard vegetation index images were output in JPG format (the type of vegetation index was set manually according to the actual situation). The fourth step was a row and column detection in the orchard canopy. The orchard point-cloud models and vegetation index images were read in Matlab, and the grayscale projection method based on 2D vegetation index images was used to realize the automatic row and column detection and segmentation of 2D/3D orchard canopy images. The point-cloud model of each fruit tree was saved in a 2D cell array. The fifth step was to calculate canopy information. According to the point-cloud model of each fruit tree, various morphological characteristic parameters were calculated, such as canopy height, canopy projection area, and circumscribed volume of the canopy. According to the vegetation index of each fruit tree, various color characteristic parameters were calculated, such as the first-, second-, and third-order color moments. Textural characteristic parameters were calculated, such as contrast, cross-correlation, energy, and inverse differential moment. The sixth step was fruit yield prediction and analysis. With the fruit tree morphological characteristic parameters, color characteristic parameters, and textural characteristic parameters as inputs, the back-propagation artificial neural network (BPANN) prediction model performed fruit yield prediction, and the UAV measurement accuracy of the morphology and yield information of the fruit trees was evaluated and analyzed.

Measurement of Multimodal Information about the Orchard Canopy
In this study, the apple orchard test site for canopy multimodal information measurement was in Yaojiazhuang, Guandao Town, Shandong Province, China (E 120.6352 • , N 37.1716 • ). The measurement test time was from September 15, 2019 to September 25, 2019, when apple trees are at the mature stage. The variety in the apple orchard was Yanfu 3, and the tree age in the measured area was 8-9 years. The measured area was 140 m × 25 m (length × width), including five rows of fruit trees; the row spacing of the fruit trees was 5 m, and; the plant spacing was 4 m. There were few dead trees or young plants in the measured area of this orchard. Because there was plenty of sunlight between 10:00 a.m. and 2:00 p.m. on a sunny day, the impact of tree shadow on image processing was relatively small, so the UAV flight time was between 10:00 and 14:00. The parameters of the Parrot Bluegrass flights are shown in Table 1. The respective flight heights were specified as 22, 32, 43, 53, and 64 m, and the ground sampling distances (GSDs) were 2.13, 3.31, 4.39, 5.43, and 6.69 cm/px. The flight speed was specified as Normal, with the averages of 2.6, 3.7, 4.9, 6.1, and 7.3 m/s. The expected flying time was 5 min 52 s, 2 min 50 s, 1 min 46 s, 1 min 32 s, and 1 min 20 s, respectively. The image front overlap was specified Sensors 2020, 20, 2985 5 of 21 as 80%, and the image side overlap was 80%. The camera was a Parrot Sequoia, which contained a 16-megapixel RGB camera and four 1.2-megapixel spectral channel single-band cameras with a global shutter. The ground control adopted a Parrot Skycontroller 2 controller and Pix4Dcapture flight control software. During the study period, the sky was clear with very few clouds and a wind speed of less than 8 m/s, which met the requirements for remote sensing by UAV. In this study, Pix4Dmapper software was used to generate 3D point-cloud models of the orchard and vegetation index orthographic images with GRE, RED, REG, and NIR orchard canopy images collected in the measured area. The main process included reading UAV measurement images, specifying coordinate systems, initial processing, and generation of point-clouds, textures, orthophotos, and indices. Pix4Dmapper parameter settings are shown in Table 2. Based on the structure-from-motion (SfM) principle [29], Pix4Dmapper carries out computing keypoints, computing matches, calibration, matching, and point-cloud densification on images, and it finally outputs the PLY file of the orchard point-cloud model, orthographic images, and various JPG files of vegetation index images.  Figure 1a shows an image of the 3D point-cloud model from the top view (XOY perspective) of the measured area of the apple orchard. Figure 1b shows the 3D point-cloud model from the side view (YOZ perspective) of the measured area in the apple orchard. Figure 1c-l, respectively, show the NG, normalized red (NR), NDVI, green NDVI (GNDVI), CIG, DVI, optimized soil-adjusted VI (OSAVI), RDVI, nonlinear VI (NLI), and wide-dynamic-range VI (WDRVI) images of the measured area in the apple orchard (grayscale of vegetation index images, convert the index value to 0-255). The formulas to calculate the vegetation indices are shown in Table 3. Table 3. Vegetation indices and their calculation formulas.

Vegetation Index Formula Vegetation Index Formula
Sensors 2020, 20, x 6 of 23  Figure 1a shows an image of the 3D point-cloud model from the top view (XOY perspective) of the measured area of the apple orchard. Figure 1b shows the 3D point-cloud model from the side view (YOZ perspective) of the measured area in the apple orchard. Figure 1c -Figure 1l, respectively, show the NG, normalized red (NR), NDVI, green NDVI (GNDVI), CIG, DVI, optimized soil-adjusted VI (OSAVI), RDVI, nonlinear VI (NLI), and wide-dynamic-range VI (WDRVI) images of the measured area in the apple orchard (grayscale of vegetation index images, convert the index value to 0-255). The formulas to calculate the vegetation indices are shown in Table 3.

Row and Column Detection Methods of the Orchard Canopy
In this study, the row and column detection method of the orchard was proposed: the row and column detection method based on grayscale projection in orchard index images (RCGP). The PLY data of a 3D point-cloud model of the orchard consist of several 3D coordinates, while the JPG data of a vegetation index is a 2D image. The data formats of the two are thus inconsistent. To achieve unified row and column segmentation across the two types of data, it is necessary to perform a conversion of segmentation positions. In this study, the segmentation performance of the RCGP was analyzed. The RCGP based on grayscale projection in orchard index images was selected, and through the row and column segmentation position conversion, the unified segmentation of the orchard multimodal data was achieved.
In the first step, a vegetation index image was grayscale processed, and the cumulative grayscale value for each row was calculated using Equation (1), while the cumulative grayscale value for each column was calculated using Equation (2). In the second step, the cumulative grayscale values for the rows were searched to find the local peaks by probability density estimation, as shown in Equation (3), based on the principle of kernel density estimation, which were taken as the row segmentation positions and were saved to the array L2D. In the third step, the cumulative grayscale values for the

Row and Column Detection Methods of the Orchard Canopy
In this study, the row and column detection method of the orchard was proposed: the row and column detection method based on grayscale projection in orchard index images (RCGP). The PLY data of a 3D point-cloud model of the orchard consist of several 3D coordinates, while the JPG data of a vegetation index is a 2D image. The data formats of the two are thus inconsistent. To achieve unified row and column segmentation across the two types of data, it is necessary to perform a conversion of segmentation positions. In this study, the segmentation performance of the RCGP was analyzed. The RCGP based on grayscale projection in orchard index images was selected, and through the row and column segmentation position conversion, the unified segmentation of the orchard multimodal data was achieved.
In the first step, a vegetation index image was grayscale processed, and the cumulative grayscale value for each row was calculated using Equation (1), while the cumulative grayscale value for each column was calculated using Equation (2). In the second step, the cumulative grayscale values for the rows were searched to find the local peaks by probability density estimation, as shown in Equation (3), based on the principle of kernel density estimation, which were taken as the row segmentation positions and were saved to the array L 2D . In the third step, the cumulative grayscale values for the columns were searched to find the local peaks, which were taken as the column segmentation positions and were saved to the cell array C 2D {i}: where I is the grayscale image of the vegetation index; (i, j) are the image coordinates; m is the number of image rows; n is the number of image columns; G L (i) is the cumulative grayscale value of the i th row; and G C (i) is the cumulative grayscale value of the j th column;f h (G L ) is the probability density value of G L ; h is the bandwidth, and; K is the kernel function.
Sensors 2020, 20, 2985 As shown in Equations (4) and (5), the row segmentation position obtained by the grayscale projection method was converted into a 3D point-cloud model, which could adopt the same row and column segmentation position, thereby realizing the unified row and column segmentation positions with the multimodal data of the orchard. Conversely, the row and column segmentation position obtained by kernel density estimation from the 3D point-cloud model was also converted into vegetation index images:

Methods for Characteristic Value Extraction for Multimodal Data of the Orchard Canopy
To measure orchard canopy information, the morphological characteristic values of fruit trees were extracted through the 3D point-cloud model of the orchard, and the canopy color and textural characteristic values were extracted through the vegetation indices of the orchard.

Calculation Method for Morphological Characteristic Values of Fruit Trees
In this study, the morphological characteristic values of fruit trees extracted mainly included height H, projected area S XOY , and volume V. The height H is the canopy height of an apple tree, and the maximum vertical height of the Z-axis in a point-cloud model is the canopy height of the highest fruit tree, as shown in Equation (6). The canopy shape of fruit trees is complex, so it is difficult to accurately calculate the canopy volume. There exists a large error in the calculation of the volume using the enclosed volume outside the 3D grids. This is mainly because though the orchard point-cloud models generated by the UAV measurement system have dense point-clouds in the XOY perspective, the point-clouds in the YOZ and XOZ perspectives and other lateral perspectives are relatively sparse. There are also other influencing factors, such as noise interference. In this study, based on the shapes of fruit trees in the measured area, the volume of the canopy was approximated using an outer enclosed ellipsoid circumscribing the canopy. As the point-cloud projection from the XOY perspective had the characteristic of rotation invariance, the point-cloud projection boundary of the XOY perspective was used to calculate the surrounding area S XOY , as shown in Equation (7). Given that the average trunk height of the fruit trees in the measured area was 0.6 m, the volume of the canopy-circumscribing ellipsoid was calculated using Equation (8). The number of canopy point-clouds P ij of each fruit tree was counted by Equation (9).
where H is the canopy height of a fruit tree, with the unit m; Z max is the maximum value on the Z-axis of the point-cloud model of any single apple tree, with the unit m; Z min is the minimum value on the Z-axis of the point-cloud model of any single apple tree, with the unit m; S XOY is the projection area of a single apple tree's canopy point-cloud model in the XOY plane, with the unit m 2 ; N is the number of convex hull vertices; x i is the x coordinate of the i th vertex; x j is the x coordinate of the j th vertex; y i is the y coordinate of the i th vertex; y j is the y coordinate of the j th vertex; x i+1 is the x coordinate of the (i + 1) th vertex; y i+1 is the y coordinate of the (i + 1) th vertex; V is the equivalent volume of the circumscribed ellipsoid of the canopy, with the unit m 3 ; P ij is the number of point-clouds in the i th row and j th column of the fruit tree canopy; C{i} (j) is the value of the initial point on the X-axis of the i th row and j th column; C{i} (j + 1) is the value of the initial point on the X-axis of the i th row and (j + 1) th column;; L(i) is the initial point on the Y-axis of the i th column; and L(i + 1) is the initial point on the Y-axis of the (i + 1) th column.

Calculation Method for the Color Characteristic Values of Fruit Trees
The extracted color characteristic values of fruit trees mainly included the first-, second-, and third-order color moments [30]. The calculation formulas are Equations (10)− (12). The color moment is a simple and effective way to express color characteristic values. Since the color information is mainly distributed in low-order moments, the first-, second-, and third-order moments are sufficient to express the color distributions in vegetation index images. There is good evidence that color moments can effectively represent the color distribution in images. The advantages of this method are that it does not require color space quantization and the characteristic vector has a low dimension.
where µ is the first-order color moment of the image; σ is the second-order color moment of the image; S is the third-order color moment of the image; p(i, j) is the color value of the image pixel at the coordinate (i, j); m is the number of rows in the image, and; n is the number of columns in the image.

Calculation Method for the Textural Characteristic Values of Fruit Trees
Grayscale symbiosis matrices or gray-level co-occurrence matrices (GLCMs) were used to express the textural characteristic values of vegetation index images. Haralick et al. [31] were the first to convert grayscale values into textural information based on GLCMs. GLCMs have the advantages of simple processing, good discrimination, and strong adaptability. They have become an important method for analyzing image textural characteristic values [31]. The characteristics of GLCMs mainly include contrast (CON), correlation (COR), angular second-order moment (ASM), and entropy; the calculation formulas are Equations (13)− (16). CON reflects the image clarity and texture depth, and its value indicates the local grayscale correlation in an image. COR is a measure of the similarity of spatial GLCM elements in the row or column direction, and its value indicates the local grayscale correlation in an image. ASM reflects the uniformity of grayscale distribution and the thickness of texture in an image. The more concentrated the grayscale distribution in an image or the coarser the texture, the greater the ASM value; conversely, the more discrete the grayscale distribution or the finer the texture, the lower the ASM value. INM reflects the homogeneity of image texture and measures the local change of image texture; the larger the inverse difference moment, the smaller the change in image texture between different regions, and the greater the local uniformity. These variables are calculated as follows: where CON is the image contrast; COR is the image correlation; ASM is the angular second moment of the image; INM is the inverse difference moment; (i, j) are two different grayscale values; p(i, j) is the normalized probability of the occurrence of pixel pair (i, j); and N is the grayscale level of the image.

Results and Analysis Methods for Row and Column Detection in the Orchard Canopy
Next, the segmentation accuracy (correction rate) and false/erroneous recognition rate of the row and column detection method based on RCGP were statistically analyzed; the calculation formulas are Equations (17) and (18). Specifically, the segmentation performance based on the NG vegetation index images was calculated when GSD was 2.13, 3.31, 4.39, 5.43, and 6.69 cm/px.
where C is the correction/accuracy rate, reported as %; NC is the number of correctly segmented regions; N is the number of total regions; E is the erroneous recognition (false recognition) rate, reported as %; and NE is the number of erroneously recognized regions (including repeat recognition regions and erroneously recognized regions).

Results and Analysis Methods for the Measurement of Orchard Morphological Characteristic Values
First, a tape measure was used to manually measure the total height of the canopy, trunk height, canopy width, and fruit yield of the apple tree in the measurement area. Each measurement was done three times, and the average value was taken. Based on the average width of a canopy, the area of canopy projection was calculated. According to Equations (6)- (8), the volume of the circumscribing ellipsoid of the apple tree canopy was calculated as the measured circumscribing volume of the apple tree canopy.
Through the methods in Section 2.4.1, the 3D point-cloud model of the orchard was extracted. The height of fruit tree H, the projected area S XOY , and the volume V were extracted. The correctness (accuracy) of fruit tree morphological characteristic values was calculated when GSD was 2.13, 3.31, 4.39, 5.43, and 6.69 cm/px. The coefficient of determination (R 2 ), root mean square error (RMSE), and relative average deviation (RAD) were used to analyze the measurement errors of fruit tree morphological characteristic values.

Results and Analysis Methods for Orchard Yield Measurement
In this study, BPANN was used for modeling. Specifically, the independent variables in the model included three morphological characteristic values, three color characteristic values, and four textural characteristic values. NG, NR, NDVI, GNDVI, DVI, CIG, OSAVI, RDVI, WDRVI, and NLI were used for extracting color characteristic values and textural characteristic values. The actual measured value of fruit yield was taken as the dependent variable. Statistical analysis was performed on the model accuracy of fruit yield measurement when GSD was 2.13, 3.31, 4.39, 5.43, and 6.69 cm/px. To evaluate the accuracy of the model and test the performance of the model in fruit yield prediction, the R 2 , RMSE, and RAD were used.

Characteristic Extraction from Multimodal Data of the Orchard Canopy
The grayscale image of the NG index was used for row and column detection in the orchard. Figure 2a shows the Y-axis grayscale projection of the orchard vegetation index image, which is the cumulative grayscale value for all the rows in the grayscale image of the index NG. The peak and valley values of the Y-axis cumulative grayscale values are marked with red and blue circles, respectively, and the local peak values of the Y-axis cumulative grayscale values were taken as the row segmentation positions. The row segmentation positions were saved into the array L; the images of each row's area were also saved. Likewise, the X-axis in the image for each row's area was in turn grayscale projected, and the local peak values in the cumulative grayscale values and their corresponding X-values were searched. As shown in Figure 2b, the local peak values of the X-axis cumulative grayscale values for each row are marked with red circles, and the local grayscale peak values of the X-axis were taken as the column segmentation positions, which were saved into the cell array C{i}.
The measured area in this study had a total of five rows, and there were, respectively, 30, 31, 32, 33, and 34 apple trees in rows 1 to row 5. This gave a total of 160 apple trees, including eight dead trees that had only a trunk or had few leaves. Figure 3 shows the row and column detection results based on grayscale projection in orchard index images. Based on the RCGP method, the row segmentation positions were accurately detected (accuracy rate 100%) under various GSDs.
In this study, the correction/accuracy rate C and the erroneous-recognition rate E were calculated with Equations (17) and (18). Statistical analysis of the RCGP method was performed, and the performance of column detection when GSD was 2.13, 3.31, 4.39, 5.43, and 6.69 cm/px was compared.
In Figure 4, when using the RCGP detection method and GSD values of 2.13, 3.31, 4.39, 5.43, and 6.69 cm/px, the average correction/accuracy rates of column detection in the grayscale image of NG index were 100.00%, 98.71%, 98.77%, 99.38%, and 100.00%, respectively, and the average erroneous-recognition rates were 1.94%, 4.42%, 3.78%, 3.76%, and 3.23%. With the RCGP detection method, the accuracy rate and misrecognition rate in column detection were not significantly affected by GSD.  The measured area in this study had a total of five rows, and there were, respectively, 30, 31, 32, 33, and 34 apple trees in rows 1 to row 5. This gave a total of 160 apple trees, including eight dead trees that had only a trunk or had few leaves. Figure 3 shows the row and column detection results based on grayscale projection in orchard index images. Based on the RCGP method, the row segmentation positions were accurately detected (accuracy rate 100%) under various GSDs.   The measured area in this study had a total of five rows, and there were, respectively, 30, 31, 32, 33, and 34 apple trees in rows 1 to row 5. This gave a total of 160 apple trees, including eight dead trees that had only a trunk or had few leaves. Figure 3 shows the row and column detection results based on grayscale projection in orchard index images. Based on the RCGP method, the row segmentation positions were accurately detected (accuracy rate 100%) under various GSDs.  According to the column detection statistics, the main reason for false recognition by RCGP lay in the misidentification of the border area. Because the orchard point-cloud model generated by the UAV was matched according to the key points of the images collected from different waypoints, there were very few leaves on the dead trees, resulting in very few point-clouds generated in these locations/areas, which in turn affected the performance of the 3D detection method. On the other hand, the RCGP detection method detected the dead trees with few leaves, so the column detection performance using this method was optimal, accurate, and stable. performance of column detection when GSD was 2.13, 3.31, 4.39, 5.43, and 6.69 cm/px was compared.
In Figure 4, when using the RCGP detection method and GSD values of 2.13, 3.31, 4.39, 5.43, and 6.69 cm/px, the average correction/accuracy rates of column detection in the grayscale image of NG index were 100.00%, 98.71%, 98.77%, 99.38%, and 100.00%, respectively, and the average erroneousrecognition rates were 1.94%, 4.42%, 3.78%, 3.76%, and 3.23%. With the RCGP detection method, the accuracy rate and misrecognition rate in column detection were not significantly affected by GSD.  According to the column detection statistics, the main reason for false recognition by RCGP lay in the misidentification of the border area. Because the orchard point-cloud model generated by the UAV was matched according to the key points of the images collected from different waypoints, there were very few leaves on the dead trees, resulting in very few point-clouds generated in these locations/areas, which in turn affected the performance of the 3D detection method. On the other hand, the RCGP detection method detected the dead trees with few leaves, so the column detection performance using this method was optimal, accurate, and stable.

Calculation and Error Analysis of Orchard Canopy Morphological Information
With the RCGP row and column detection method, the row and column segmentation positions in vegetation index images were determined. The row and column segmentation positions in the 3D point-cloud model were obtained using Equations (4) and (5). The unified segmentation positions in the fruit tree region were used. The morphological parameters of the fruit tree canopy were calculated using Equations (6)-(9), the color characteristic values of the vegetation indices of the fruit tree canopy were calculated using Equations (10)- (12), and the textural characteristic values of the vegetation indices of the fruit tree canopy were calculated using Equations (13)- (16). Figure 5a shows the detection results for the fruit tree canopy height H, where the highest point and lowest point in each fruit tree area were detected, and the distance between the two points was the height of the fruit tree, as shown in Figure 5b. Figure 5c shows the detection results for SXOY, the projected area of the fruit tree canopy, which is the area enclosed by the projected boundary of the point-cloud model in the XOY perspective, the red circle and the column mark the boundary convex hull point set. Taking the coordinates with average values of the X-axis and Y-axis of the point-cloud model as the center point, the equivalent circle of canopy projection was drawn (the black circle in Figure 5c). The area of this circle is equal to SXOY. In this study, the average value of the trunk of the fruit trees was 0.6 m, and the shape of the canopy was similar to that of an ellipsoid. Hence, based on the tree height H, the height of the trunk, and the projected canopy area SXOY, the volume of the canopy-circumscribing ellipsoid of fruit tree was calculated, as shown in Figure 5d.

Calculation and Error Analysis of Orchard Canopy Morphological Information
With the RCGP row and column detection method, the row and column segmentation positions in vegetation index images were determined. The row and column segmentation positions in the 3D point-cloud model were obtained using Equations (4) and (5). The unified segmentation positions in the fruit tree region were used. The morphological parameters of the fruit tree canopy were calculated using Equations (6)-(9), the color characteristic values of the vegetation indices of the fruit tree canopy were calculated using Equations (10)- (12), and the textural characteristic values of the vegetation indices of the fruit tree canopy were calculated using Equations (13)- (16). Figure 5a shows the detection results for the fruit tree canopy height H, where the highest point and lowest point in each fruit tree area were detected, and the distance between the two points was the height of the fruit tree, as shown in Figure 5b. Figure 5c shows the detection results for S XOY , the projected area of the fruit tree canopy, which is the area enclosed by the projected boundary of the point-cloud model in the XOY perspective, the red circle and the column mark the boundary convex hull point set. Taking the coordinates with average values of the X-axis and Y-axis of the point-cloud model as the center point, the equivalent circle of canopy projection was drawn (the black circle in Figure 5c). The area of this circle is equal to S XOY . In this study, the average value of the trunk of the fruit trees was 0.6 m, and the shape of the canopy was similar to that of an ellipsoid. Hence, based on the tree height H, the height of the trunk, and the projected canopy area S XOY , the volume of the canopy-circumscribing ellipsoid of fruit tree was calculated, as shown in Figure 5d.
Statistical analysis was performed on the measurement results of morphological parameters of the fruit tree canopy, including H, S XOY , and V, as shown in Table 4 and RADavg was 7.90%, 8.28%, 11.90%, 12.61%, and 13.69%. Thus, the statistical results showed that GSD had a significant impact on the accuracy and correlation of the calculated 3D morphological parameters in the point-cloud model of the orchard: as GSD increased, R 2 decreased and RMSE increased. When GSD was 2.13, 3.31, 4.39, 5.43, and 6.69 cm/px, the point-cloud density in the 3D point-cloud model was, respectively, 364.77 points/m 3 , 75.55 points/m 3 , 32.01 points/m 3 , 17.22 points/m 3 , and 8.65 points/m 3 . As GSD directly affected the density of the orchard point-cloud model, the average point-cloud density had a significant decreasing trend as GSD increased, resulting in an increasing error in the measurement of canopy morphological parameters. GSD had a greater impact on the measured values of SXOY and V, and a small amount of noise caused a large error in The S XOY of the fruit tree canopy had a higher measurement error than H, which was mainly because the S XOY values were calculated from the actual measured widths of fruit trees. Although the width of the fruit tree was the average of multiple measurements, the width of a fruit tree could easily be measured wrong by hand, leading to a certain difference from the calculated value. In the UAV measurement method, as long as the fruit tree area is accurately segmented, the canopy boundary point set can be quickly found through the boundary point spacing search method. Although S XOY can be calculated from the boundary convex hull point set, it requires an accurate point-cloud model. The fruit tree canopy V had a higher measurement error than H and S XOY , which was mainly due to the use of canopy-circumscribing ellipsoids to approximate the canopy volume of fruit trees. There were errors in both UAV measurements and manual measurements of H and S XOY , which led to a relatively large error in V.

Orchard Yield Prediction and Error Analysis
Three color moment characteristic values and four textural characteristic values of the fruit tree canopy vegetation indices and four morphological characteristic values of the 3D point-cloud model were used as the input characteristic variables in the orchard yield prediction model. The actual measured yield values of fruit trees were taken as the model output variables for constructing the fruit yield prediction model. The canopy vegetation indices of fruit trees were NG, NR, NDVI, GNDVI, DVI, CIG, OSAVI, RDVI, WDRVI, and NLI (Table 3) BPANN was used to establish the fruit yield prediction model. A three-layer network structure was used, where the excitation function from the input layer to the hidden layer was a tansig function and that from the hidden layer to the output layer was a purelin function. The learning function was a function of gradient descent with momentum weight, and the training function was the Levenberg-Marquardt algorithm. The empirical formula h = sqrt(m + n) + a was used to calculate and select the node number of the hidden layer, where h is the number of hidden layers in the BPANN, m is the number of input layers, n is the number of output layers, and a is the adjustment constant (a = 1-10). The percentages of training, validation, and testing data of BPANN model is 60%, 15%, and 25%, respectively. The performance of the fruit yield prediction model under different GSDs was statistically analyzed.
When GSD was 2.  The statistical data showed that when the color moment characteristic values, textural characteristic values, and 3D morphological characteristic values of one vegetation index were input into the yield prediction model, the correlation between the predicted yield value and the measured value was not significant, and the differences in the prediction performance between various vegetation indices were not significant. The main reason was that one vegetation index has limited information, and the correlation between fruit yield and fruit tree morphology is not significant enough, which led to the inadequate performance of the fruit yield prediction model.
respectively. The performance of the fruit yield prediction model under different GSDs was statistically analyzed.
When GSD was 2.13, 3.31, 4.39, 5.43, and 6.69 cm/px, for Input 1, in which the characteristic values of vegetation indices NG, NR, NDVI, GNDVI, DVI, CIG, OSAVI, RDVI, WDRVI, and NLI were separately used in the BPANN prediction model, R 2 ranged from 0.59-0.73, 0.54-0. The statistical data showed that when the color moment characteristic values, textural characteristic values, and 3D morphological characteristic values of one vegetation index were input into the yield prediction model, the correlation between the predicted yield value and the measured value was not significant, and the differences in the prediction performance between various vegetation indices were not significant. The main reason was that one vegetation index has limited information, and the correlation between fruit yield and fruit tree morphology is not significant enough, which led to the inadequate performance of the fruit yield prediction model.  Statistical analysis showed that when the fruit tree morphological characteristic values were used to predict the fruit yield, the model correlation was not significant, and as GSD increased, the performance of this prediction model worsened, with an increase in the RAD. This was mainly because GSD affected the accuracy of the orchard 3D point-cloud model, which led to an increase in the calculation error of the morphological characteristic values; additionally, the morphological characteristic values are not the only determinants of fruit yield.
Statistical analysis showed that when the fruit tree morphological characteristic values were used to predict the fruit yield, the model correlation was not significant, and as GSD increased, the performance of this prediction model worsened, with an increase in the RAD. This was mainly because GSD affected the accuracy of the orchard 3D point-cloud model, which led to an increase in the calculation error of the morphological characteristic values; additionally, the morphological characteristic values are not the only determinants of fruit yield.   When the GSD was 2.13, 3.31, 4.39, 5.43, and 6.69 cm/px, for Input 3, the R 2 of the BPANN prediction model was, respectively, 0.88, 0.83, 0.86, 0.75 and 0.83, as shown in Figure 7a; the RAD values were respectively 8.74%, 9.01%, 8.56%, 12.15% and 10.17%, as shown in Figure 7b; and the RMSE values were respectively 7.71 kg, 7.94 kg, 7.20 kg, 9.69 kg and 8.29 kg. The statistical data showed that when the color characteristic values of fruit tree vegetation indices were used to predict the fruit yield, the model correlation was significant; only when the GSD was 5.43 cm/px, the significance was relatively low. Under different GSDs, the performance of the prediction model was similar. The performance of the prediction model outperformed that of Input 1, where single index characteristic values were used for model prediction. The color characteristic values of the fruit tree canopy vegetation index can be used to predict the yield of fruit trees.
When GSD was 2.13, 3.31, 4.39, 5.43, and 6.69 cm/px, for Input 4, the R 2 of the BPANN prediction model was, respectively, 0.56, 0.57, 0.58, 0.66, and 0.66 ( Figure 7a); RAD was 16.03%, 17.59%, 13.66%, 12.91%, and 12.47% (Figure 7b); and RMSE was 13.13 kg, 12.78 kg, 12.71 kg, 11.25 kg, and 12.10 kg. The statistical analysis showed that when the textural characteristic values of fruit tree vegetation indices were input to predict the fruit yield, the model correlation was not significant. As GSD increased, the prediction model performance improved. This was mainly because GSD affects the textures of orchard vegetation index images, so as GSD increased, the canopy area in the vegetation index image became more blurred, while the edges became clearer. However, using the textural characteristic values of vegetation indices to predict the yield of fruit trees has a relatively large error, so textural characteristic values cannot be applied to yield prediction of fruit trees.
When GSD was 2.13, 3.31, 4.39, 5.43, and 6.69 cm/px, for Input 5, the R 2 of the BPANN prediction model was, respectively, 0.88, 0.88, 0.84, 0.83, and 0.83 ( Figure 7a); RAD was 8.57%, 7.85%, 9.23%, 10.23%, and 10.10% (Figure 7b); and RMSE was 8.35 kg, 6.74 kg, 8.18 kg, 8.62 kg, and 8.28 kg. The statistical analysis showed that when the color characteristic values and textural characteristic values of fruit tree vegetation indices were input to predict the fruit yield, the model correlation was significant; as GSD increased, the performance of this prediction model worsened. The model performance was similar to that of Input 3. Hence, the color characteristic values and textural characteristic values of canopy vegetation indices can be used to predict fruit yield, but considering the prediction performance, textural characteristic values can be ignored, and the color characteristic values of canopy vegetation indices can be used by themselves to predict fruit yield.
When GSD was 2.13, 3.31, 4.39, 5.43, and 6.69 cm/px, for Input 6, the R 2 of the BPANN prediction model was, respectively, 0.88, 0.87, 0.86, 0.83, and 0.84 ( Figure 7a); RAD was 8.10%, 8.05%, 9.03%, 9.56%, and 9.76% (Figure 7b); and RMSE was 6.90 kg, 7.03 kg, 7.25 kg, 7.96 kg, and 7.93 kg. The statistical analysis showed that when the color characteristic values and textural characteristic values of fruit tree vegetation indices and the morphological characteristic values of the point-cloud model were input to predict fruit yield, the model correlation was significant, and the prediction model performance was similar to that of Input 3 and Input 5. Therefore, the input characteristics of Input 6 can be applied to fruit yield prediction.
Based on the above analysis, using the color characteristic values of canopy vegetation indices of fruit trees to predict the yield of fruit trees performed better than using the textural and morphological characteristic values. This was mainly because given the UAV measurement mode used in this study, the generated orthophoto images of orchard canopy had a large amount of color characteristic information, while the textural characteristic values of the canopy were relatively blurred, so the amount of information they bore was relatively small. On the other hand, the morphological characteristic values were affected by the accuracy of the 3D point-cloud model, and the morphological characteristic values could not adequately predict the fruit yield. Thus, using color characteristic values of multiple vegetation indices to predict yield performance is superior to using that of one vegetation index, which is mainly due to the screening of vegetation indices in this study, in which vegetation indices with more obvious characteristic advantages were selected. Multiple vegetation indices contained more canopy information and thus had better prediction performance. This method can be applied to yield prediction of any modern standardized apple orchards.

Conclusions
This study proposes a UAV multimodal information-based method for the measurement of canopy morphological information and for yield prediction of apple orchards. The main conclusions are as follows: 1.
In this study, a row and column detection method based on grayscale projection in orchard index images, RCGP, is proposed. It allows row and column segmentation using multimodal information of fruit tree canopies in modern standardized apple orchards. The results showed that using the RCGP method, the correction/accuracy rate of row detection in the orchard was 100.00%. Using the RCGP method, when GSD was 2.13, 3.31, 4.39, 5.43, and 6.69 cm/px, the average correction/accuracy rates of column detection based on the grayscale images of NG index were, respectively, 100.00%, 98.71%, 98.77%, 99.38%, and 100.00%, and the average misrecognition rates were 1.94%, 4.42%, 3.78%, 3.76%, and 3.23%. The RCGP detection method can detect dead trees with few leaves, so the column detection performance using this method was accurate and stable.

2.
A method for measuring canopy morphological information of fruit trees based on the 3D point-cloud model of orchards is established. The model correlation was significant, which can be applied to fruit yield prediction.
The row and column detection method proposed in this study is suitable for segmentation using multimodal canopy data in modern standardized orchards, which lays a good foundation for orchard canopy information measurement. The information of canopy morphological and vegetation index characteristic values could be extracted at the same time by this method, which provided more effective information for fruit yield prediction and improves the accuracy of fruit yield prediction. This study established a canopy morphological measurement method and a yield prediction method for fruit trees, which can provide a theoretical basis and technical reference for canopy information measurements in modern standardized orchards and will be valuable in the production management of modern standardized orchards.