Next Article in Journal
Application of a Generic Participatory Decision Support System for Irrigation Management for the Case of a Wine Grapevine at Epirus, Northwest Greece
Previous Article in Journal
Genetic Components of Self-Incompatibility in Brassica Vegetables
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Horticulturae
Volume 9
Issue 2
10.3390/horticulturae9020266
horticulturae-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

UAV Photogrammetry-Based Apple Orchard Blossom Density Estimation and Mapping

by
Wenan Yuan
1,*,
Weiyun Hua
2,
Paul Heinz Heinemann
2 and
Long He
2,3
1
Independent Researcher, Burr Ridge, IL 60527, USA
2
Department of Agricultural and Biological Engineering, The Pennsylvania State University, University Park, State College, PA 16802, USA
3
Fruit Research and Extension Center, The Pennsylvania State University, Biglerville, PA 17037, USA
*
Author to whom correspondence should be addressed.
Horticulturae 2023, 9(2), 266; https://doi.org/10.3390/horticulturae9020266
Submission received: 13 January 2023 / Revised: 13 February 2023 / Accepted: 13 February 2023 / Published: 16 February 2023
(This article belongs to the Section Floriculture, Nursery and Landscape, and Turf)
Browse Figures
Versions Notes

Abstract

:
Thinning is an important routine for apple growers to manage crop load and improve fruit quality, which can be accomplished through manual, chemical, or mechanical manipulation of flowers and fruitlets. Traditionally, blossom thinning relies on human experts’ visual evaluation of the flower load, a leading indicator of crop load, which can be imprecise and prone to errors. This study aimed to develop an apple blossom density mapping algorithm utilizing point clouds reconstructed through unmanned aerial vehicle (UAV)-based red-green-blue (RGB) imagery and photogrammetry. The algorithm was based on grid average downsampling and white color thresholding, and it was able to generate top-view blossom density maps of user-defined tree height regions. A preliminary field experiment was carried out to evaluate the algorithm’s accuracy using manual blossom counts of apple tree row sections as ground truths, and a coefficient of determination (R2) of 0.85, a root mean square error (RMSE) of 1307, and a normalized RMSE (NRMSE) of 9.02% were achieved. The algorithm was utilized to monitor the blooming of the apple tree rows and was demonstrated to effectively show blossom density variations between different tree rows and dates. The study results suggested the potential of UAVs as a convenient tool to assist precise blossom thinning in apple orchards, while future research should further investigate the reliability of photogrammetry techniques under different image qualities and flight settings as well as the influence of blossom distribution on algorithm accuracy.

1. Introduction

Apple (Malus domestica) is one of the most valuable and culturally significant fruits worldwide [1]. For the crop year 2022/2023, the global apple production was forecast to be 78.8 million tons [2]. In the United States, more than 5000 farms with almost 85,000 acres of apples are located in the northeastern region, and more than 200,000 tons of apples can be produced in Pennsylvania per year [3]. Although apple has been a domesticated crop for thousands of years, challenges still exist in fruit production management. For example, apple trees tend to overproduce flowers, which can grow into fruits with inadequate sizes and less marketable values due to insufficient resource allocation. Following the years with large yields of small-sized fruits, certain apple varieties usually have low or even no yields in the following years, which is known as the alternate bearing or biennial bearing phenomenon [4]. In order to achieve stable production of apples with high quality year after year, human interventions are needed for trees to have a balanced ratio between canopy photosynthesis and fruit load [5].
Thinning is a crucial crop load management practice for growers to increase fruit size and quality [6], suppress alternate bearing for consistent yields over the years, and reduce tree limb breakage. The current fruit tree thinning methods include manual, chemical, and mechanical thinning. Manual or hand thinning is a labor-intensive and time-consuming process, during which fruitlets are manually removed typically after the physiological fruit drop based on their size and proximity to one another [7]. High demand for manpower and reduced thinning benefit due to late execution time are the two major drawbacks of manual thinning [7]. During chemical thinning, chemical substances are sprayed onto trees at one or more dates to prevent flower pollination or fertilization, or stimulate fruitlet drop [8]. The efficacy of chemical thinning can be influenced by many factors, such as cultivar, rootstock, pollen density, weather, thinner chemistry, and application method, rate, and timing [9], while predictive models, such as the pollen tube growth model [10], carbohydrate model [11], and fruitlet growth model [12], have been proposed and utilized to achieve more precise thinner applications. Despite early blossom thinning having the greatest potential in improving fruit size, quality, and color at harvest and promoting return blooming [13], the unpredictability of chemical thinning has been an issue for apple growers for more than 50 years [14]. Mechanical thinning works by physically removing a portion of flowers or fruitlets using mechanical devices, such as spinning rotors with wires attached. It has the advantage of having no chemical impact on the environment and lower thinning time and cost, as well as being less restricted by weather compared to manual and chemical thinning. Various mechanical thinning devices have been developed and evaluated in apple orchards in previous studies [15,16,17], although currently, they are non-selective and may impart damage to flowers, branches, and leaves.
Tree blooming intensity or the percentage of open blooms often needs to be estimated for blossom thinning planning, such as the timing of chemical thinning or the strength of mechanical thinning [16]. Human experts usually estimate this visually, which can be inaccurate and imprecise, as typically only a limited number of trees are inspected [7]. Among the many instruments employed in precision agriculture [18,19], unmanned aerial vehicles (UAVs) are considered an essential future technology [20] and show great potential in rapid, flexible plant data collection with high spatiotemporal resolution [21,22,23]. Applications of UAVs in agricultural studies include vegetation growth and health monitoring [24,25], yield estimation, pest management, irrigation management, and crop spraying [18]. Along with machine learning and vegetation index calculation, photogrammetry is a technique that is commonly employed for processing UAV imagery [26]. Unlike expensive active sensors, such as LiDARs [27], ToF cameras [28], and ultrasound systems [29], passive photogrammetry algorithms, such as structure from motion (SfM) [30,31] and multi-view stereo (MVS) [32], can recover 3D structures and create refined 3D models of stationary scenes from overlapping images taken by inexpensive visual sensors, such as red-green-blue (RGB) cameras. With the growing research interests in UAV applications in agriculture, advanced commercial photogrammetry software has become a standard drone mapping tool to assist researchers in areas such as crop structural parameter estimation [33], multispectral crop surveying [34], and crop health monitoring [35]. In terms of apple blossom density estimation, the combination of UAV, photogrammetry, and proper data processing algorithms can potentially not only allow farmers to monitor large-scale orchards conveniently with precision but also eliminate the need for trained human laborers to evaluate blooming intensity of trees when foreign labor supply to farms in the US is declining [36].
Several studies can be found in the current literature regarding apple blossom detection and counting using modern technology for accurate thinning or yield prediction. RGB cameras [7,37,38,39,40,41,42,43,44,45] and multispectral cameras [46] have been deployed for identifying apple flowers. Variations of data collection methods among the studies do exist. Manually capturing apple blossom images with handheld cameras [38,39,40,43] ensures high image quality; however, it is also the most laborious and least efficient method. Ground mobile units, such as utility vehicles and tractors, were used in many studies [37,39,41,44,45] as the platform for mounting cameras, artificial lights, and GPS receivers, and side-view images were taken automatically while the mobile units moved along apple tree rows. Compared to manual image capturing, ground vehicles offer a higher level of autonomy and efficiency. Yet, the work involved in developing the specialized carts, mounting structures, and sensor control programs, and keeping the platforms moving in a slow, steady manner via driving or remote control may prevent growers from adopting this method for managing large orchards. As the most efficient and user-friendly method, UAVs have been applied in research to capture top-view apple blossom images [42,46,47]. However, some of the unique challenges during UAV data collection include weather limitations (e.g., wind, rain), inconsistent image quality due to tree movement, and fewer image details due to high flight altitudes. Nonetheless, when weather allows, commercial miniature UAVs that are robust and economical demonstrate much potential in being employed by growers for orchard management [47,48,49,50,51,52,53].
Among the reviewed studies, computer-vision-based image analysis techniques were commonly utilized for apple blossom identification, from simple but effective color segmentation [42,44,45] to popular convolutional neural network (CNN)-based object detection [7,37,38,39,40,41,43,47]. However, an attempt to explore point-cloud-based apple blossom density estimation is missing in the current literature. Three-dimensional point clouds by nature store more comprehensive information, such as volume, than images, and they suffer less from issues such as object occlusion, since images used for point cloud reconstruction are captured from different camera positions. Deep-learning methods require much manual work for image annotation and high-level understanding of model training and tuning, which makes research-level CNN algorithms potentially less practical and desirable for growers than color thresholding techniques, which are simple to understand and modify.
With the goal of assisting apple growers in the decision making of precise blossom thinning, the objectives of the current study were to: (1) develop a top-view apple blossom density mapping algorithm with definable tree height regions, utilizing orchard point clouds reconstructed with UAV-based RGB images and photogrammetry software; (2) conduct a preliminary field experiment to evaluate the algorithm’s accuracy in terms of blossom counts; (3) investigate the algorithm’s utility in apple orchard blossom density monitoring.

2. The Proposed Mapping Algorithm

Figure 1 shows the overall structure of the proposed algorithm, whose principle can be summarized as follows. Given a blooming apple orchard point cloud reconstructed by a photogrammetry software, a rasterized terrain map is first generated from the point cloud. During the process, ground pixels are distinguished from tree pixels utilizing the tall height feature of trees compared to ground, interpolated to form a complete map, and smoothed to remove high frequency noise. Next, an apple blossom point cloud is extracted from the orchard point cloud based on the white color trait of blossoms. The blossom point cloud then needs to be processed by a few steps, including height adjustment using the terrain map to eliminate terrain unevenness, noise removal through height thresholding to delete off-tree white points, downsampling to unify point cloud density, and region extraction if the blossom density of a specific tree height section needs to be inspected. Finally, rasterized blossom containing volume map and blossom count map are derived from the height-adjusted, noise-removed, downsampled blossom point cloud based on its point number and geometry information, and a blossom density map is subsequently calculated as the ratio between blossom count and blossom containing volume. A detailed algorithm breakdown and explanation can be found in Appendix A. The mapping algorithm was coded in MATLAB R2021a (Natick, MA, USA).

3. Preliminary Field Experiment

As mentioned in Appendix A, the purpose of conducting a field experiment in an apple orchard was to determine the optimal white color threshold for blossom points, the ideal blossom grid filter size for downsampling blossom point clouds, and the relationship between the number of downsampled blossom points and the number of actual blossoms. However, it is beyond the scope of the study to conduct a thorough field examination of the blossom density mapping algorithm in terms of its accuracy and reliability against various factors. Rather, the results of the field experiment serve as a feasibility assessment of the proposed algorithm.

3.1. Data Collection

The field experiment was conducted in an apple orchard at Russell E. Larson Agricultural Research Center (40.708756° N, 77.953668° W), which was different from the one where the sample RGB data were collected. The new orchard was intentionally chosen to indirectly test the generalizability of the proposed mapping algorithm. The new apple orchard consisted of eight rows of trees and five cultivars, including Gibson Golden Delicious, Spur Red Delicious, Jonamac, Jonagold, and Ace Spur Red Delicious. The orchard was roughly 70 m long and 20 m wide.
To calibrate the relationship between downsampled blossom point number and blossom count, each of the eight tree rows was divided into three sections, including a 10 m section, a 20 m section, and a 40 m section, and in total, twenty-four tree row sections were utilized for ground truthing. Tarp strips were prepared and placed across the orchard to mark the tree row sections (Figure 2a), which could be easily observed by UAV during data collection. On 3 May 2022, when most of the apple trees were either at or approaching peak bloom, manual blossom counts of the 24 tree row sections were collected. During the blossom counting process, two people were required, and each person only counted one side of the tree row sections (Figure 2b).
As the transition of apple flower buds between different growth stages is gradual, it is difficult to clearly define a set of objective rules to quickly determine whether a flower bud is at the bloom stage or not, especially when blossom distribution is very dense. To facilitate efficient blossom counting, the general rule that was adopted during the manual blossom counting process was to classify a flower bud or a cluster of flower buds as flowers when they were of a substantial size and white enough to be easily recognized as flowers. Pinkish blossoms, small, unopened blossoms, and wilted blossoms with shrunken sizes were not counted during the field experiment (Figure 3).
To test the generalizability of the concept of UAV photogrammetry-based blossom density mapping, different camera and photogrammetry software from those used in Appendix A.1 were employed for the field experiment. RGB image data were collected using a DJI Mavic 2 Zoom, which had a built-in RGB camera with a 4000 × 3000 resolution and a 2× optical zoom capability. Autonomous UAV flights were conducted again using DJI GS Pro with a preplanned flight path (Figure 4a). The flight altitude was set at 15 m, and the UAV speed was set at 1 m/s. Images of the apple orchard were captured every two seconds with a 40° camera pitch and the camera’s full zoom capability, resulting in roughly a 90% overlap between consecutively collected images. Another commercial photogrammetry software, DroneDeploy (San Francisco, CA, USA), was used to reconstruct orchard point clouds (Figure 4b). RGB data of the orchard were collected on 3, 5, 8, 11, and 14 May 2022 to monitor the tree blossom density changes with time.

3.2. Data Processing and Analysis

Each RGB dataset collected was used to reconstruct an orchard point cloud model. For the orchard point cloud of 3 May 2022, when the apple blossoms were manually counted, each of the 24 tree row sections was manually cropped out using the open-source software MeshLab 2020.07 (ISTI-CNR, Pisa, Italy). Since there were two parameters that needed to be investigated, namely white blossom point color threshold and blossom grid filter size, various value combinations of the two parameters were studied. A white color threshold range of 0.01 to 0.2 with a step of 0.01 and a blossom grid filter size range of 0 to 0.23 m3 with a step of 0.013 m3 were defined. In total, there were 420 unique value combinations, and each combination was used to extract downsampled blossom points from the 24 cropped tree row section point clouds, respectively (Appendix A.3). The relationships between the extracted downsampled blossom point numbers and the manual blossom counts were analyzed using linear regression. However, the regressions were forced to have no intercept, since ideally, zero white blossom point in reconstructed orchard point clouds should indicate zero blossoms in the orchards. The 420 regressions were evaluated in terms of the coefficient of determination (R2) and root mean square error (RMSE), and the optimal combination of white color threshold and blossom grid filter size was selected based on both metrics.

4. Results and Discussion

4.1. Optimal White Blossom Color Threshold and Blossom Grid Filter Size

Figure 5 shows the R2 and RMSE heatmaps of the 420 regressions, respectively. Strong, positive linear relationships existed between the downsampled blossom points and the manual blossom counts, which proves the validity of point-cloud-based apple blossom density estimation. The largest R2 of 0.85 and the smallest RMSE of 1307 were achieved when the white color threshold was 0.04 and the blossom grid filter size was 0.093 m3. As shown in Figure 5, when the white color threshold ranged from 0.03 to 0.07 and the blossom grid filter size ranged from 0.013 m3 to 0.183 m3, most value combinations of the two parameters had relatively high R2s and low RMSEs.
The proposed mapping algorithm relied on white color thresholding, an essential and effective step, to extract blossom points from an orchard point cloud. However, there are two major sources of errors to this method: non-blossom points and white color threshold level. White color is not unique to only apple blossoms in orchards. Highly reflective surfaces, man-made objects, snow, and overexposed pixels can all appear white in RGB images. While automatic camera settings generally work well in capturing images with high dynamic ranges, lowering camera exposure can potentially be beneficial in reducing non-blossom points in a blossom point cloud. Aside from the number of non-blossom white points contained in an orchard point cloud, the efficacy of white color thresholding is also dependent on the chosen saturation and intensity threshold levels. When images are captured under different lighting conditions or at different times of the day, the same white color threshold might not always result in optimal blossom point separation. For example, for point clouds that are reconstructed using images captured at dawn or dusk when brightness is lacking, lower intensity thresholds are likely to be more effective for blossom point extraction. Potentially, image preprocessing techniques, such as contrast stretching before point cloud reconstruction, can help not only achieve more consistent image quality across datasets but also eliminate the need for establishing optimal white color thresholds for different environmental lighting conditions. As most apple varieties have pink-colored flower buds during the first and full pink stages that gradually turn white during the first and full bloom stages, pink color hue and low saturation thresholding can be another approach to separate apple blossom points from an orchard point cloud.
The presumption that downsampled blossom points can be used to reliably estimate blossom counts requires that most apple flowers have a uniform size at the time of data collection. During a growing season, the petals of apple flowers, which are the most visible portion of the flowers, transition from being closed (Figure 6a) to being open (Figure 6b), and to being wilted and detached from the flowers (Figure 6c). The size of apple flowers during this process can change substantially, which will lead to less accurate point-cloud-based blossom count estimations, considering that, usually, not all flower buds on apple trees are at the identical physiological growth stage. Aside from the developmental stage, apple variety is another factor, which can contribute to inconsistent blossom size. Calibrating unique relationships between downsampled blossom point numbers and blossom counts for individual apple varieties at peak bloom has the potential to improve blossom count estimation accuracy, which, however, might not be necessary when growers only have one or a few apple varieties in their orchards.

4.2. Blossom Count Estimation Accuracy

Figure 7 shows the scatter plot between downsampled blossom point number and manual blossom count when the white color threshold was 0.04 and the blossom grid filter size was 0.093 m3. As mentioned in Section 4.1, an R2 of 0.85, an RMSE of 1307, and a normalized RMSE (NRMSE) of 9.02% were achieved. While the R2 verified the strong positive correlation between downsampled blossom point number and blossom count, the RMSE indicated that the methodology accuracy can be potentially further improved. Nonetheless, given the large scale of manual blossom count, the NRMSE suggested the validity of the proposed mapping algorithm.
There were many factors during the field experiment, which might have influenced the results in Figure 7. First, ground truth or manual blossom counts might not be very accurate. Due to the dense distribution of apple flowers, accurately and precisely counting flowers of eight 70 m long apple tree rows at full bloom stage would be extremely laborious and time consuming, which was not practical and feasible considering the scope and scale of the study. During the manual counting, to speed up the counting process, flower clusters were often counted rather than individual flowers, and each flower cluster was assumed to contain five flowers, which might not be true for all flower clusters. Additionally, flowers of apple tree rows essentially cannot be counted by a single person without using destructive methods, as flowers growing on one side of a tree row often cannot be seen from the other side. When multiple people are involved in the counting process, accurately tracking which flowers are counted is a major challenge. It is very likely that during the field experiment, many apple flowers were either not counted at all or counted twice by the two people from two sides of the tree rows.
Second, as discussed in Section 4.1, at any given time during a growing season, it is not likely that all flowers of apple trees are at the exact developmental stage. For example, some flowers may already start to wilt, while some are yet to be fully open. The natural, inevitable varying flower size in the orchard was an important factor, which negatively affected the accuracy of the proposed methodology. The different apple tree varieties also had slightly different peak bloom dates, which contributed to the overall flower size variation in the orchard during data collection.
Third, photogrammetry techniques have limited abilities in reconstructing small objects. In order to obtain high-quality reconstructed orchard point clouds, ideally, UAVs need to fly at high altitudes to prevent significant image perspective distortions, so that more visual similarities between images can be preserved. However, as cameras are located farther away from apple trees, small apple flowers will no longer appear as prominent features in images, and often, photogrammetry software would fail to reconstruct fine details, such as small apple flowers. This explains why for low-resolution cameras, flower clusters at full bloom stage would be easier to reconstruct than individual flowers at king bloom stage due to their larger overall white mass. While high UAV flight altitude is beneficial for large-scale UAV mapping, for this specific application, the UAV flight altitude was carefully chosen to achieve a balance between image similarity and image detail. Nevertheless, the failure to reconstruct small blossoms could be one of the factors that affected the results.
Lastly, as the flowers inside apple tree canopies have a lower chance of being observed by a UAV, occlusion of flowers can lead to lower blossom count estimation accuracy. Depending on the apple variety, training system, and growth stage, dense tree canopies and dense flower distributions can both increase the number of occluded flowers. During the field experiment, all apple blossoms were counted regardless of their locations within the tree canopies, while during point cloud reconstruction, perhaps only a limited number of flowers located inside of the tree canopies were successfully reconstructed. Such mismatch, especially when multiple apple varieties were involved, might be partially responsible for the imperfect results in Figure 7. Generally, apple flowers located in upper outer tree canopies are more likely to be successfully reconstructed than those located in lower inner tree canopies, and deliberately designing UAV flight paths with appropriate flight altitude, camera pitch angle, and flying speed to maximize blossom point reconstruction success rate is vital for the proposed blossom count and density estimation methodology.

4.3. Blossom Density Monitoring Application

As a demonstration, the proposed algorithm was utilized to generate blossom density maps of the apple orchard during the blooming period using a 0.3 m × 0.3 m × 0.3 m terrain grid filter (Figure 8). From left to right, the varieties of the eight apple tree rows in the maps are Gibson Golden Delicious, Spur Red Delicious, Jonamac, Jonagold, Gibson Golden Delicious, Gibson Golden Delicious, Ace Spur Red Delicious, and Ace Spur Red Delicious. Substantial blossom density variations within the tree rows, between the tree rows and varieties, and across the dates can be easily observed in the maps, which indicates the strong potential of the proposed methodology in helping apple growers monitor the blossom density of large-scale orchards with ease.
The functions of blossom density maps are three-fold. First, they show the absolute and relative blossom densities of different apple varieties. For example, relatively speaking, Jonamac and Gibson Golden Delicious had higher peak blossom densities than Spur Red Delicious and Ace Spur Red Delicious. During blossom thinning, growers can focus more on the warm regions with high blossom densities than the cold regions with low blossom densities in a map to achieve uniform blossom density distribution over the whole orchard. Second, they capture the peak bloom date of different apple varieties. For example, the peak bloom date of Jonamac was 5 May, and the peak bloom date of Gibson Golden Delicious was 8 May. Historical blossom density maps from previous years would allow growers to better anticipate the peak bloom dates of their apple trees and prepare for blossom thinning ahead of time. Third, they record the peak bloom duration of different apple varieties. For example, the peak bloom duration of Jonagold was roughly four days, and the peak bloom duration of Gibson Golden Delicious was roughly seven days. It might be beneficial for growers to prioritize the blossom thinning of apple varieties with short peak bloom durations, as they are more time sensitive.

4.4. Implications and Future Work

The proposed blossom density mapping algorithm can assist apple growers in quickly identifying tree row regions with undesirably dense flower distributions, and hence, making informed, timely blossom thinning decisions. Compared to traditional manual orchard inspection, the algorithm allows the blossom densities of apple trees to be compared in a more objective way. Even though the absolute blossom densities estimated by the algorithm might not be particularly precise, the relative blossom densities between tree rows can be clearly reflected in the maps, which enables growers to quickly locate orchard regions requiring thinning operations. The algorithm is also helpful when orchard blossom density needs to be re-evaluated. For example, after a rainstorm or mechanical blossom thinning, a UAV flight mission can be conducted over an orchard to assess its current blossom density to help determine whether further blossom thinning is still necessary.
The primary focus of the current study was to develop a point-cloud-based apple blossom density estimation algorithm, based on the presumption that downsampled blossom point number and actual blossom count are positively correlated. While the preliminary field experiment confirmed the algorithm’s validity and general utility, to achieve more accurate blossom density estimations, comprehensive experiments exploring various factors that can influence the accuracy and repeatability of the proposed methodology are still needed.
To reconstruct high-quality orchard point clouds, a robust photogrammetry software is essential. Currently, various commercial and open-source photogrammetry software are available. Aside from PIX4Dmapper and DroneDeploy which were tested in the study, Agisoft Metashape, COLMAP, Meshroom, OpenMVG, and VisualSFM are also some of the available photogrammetry software [54,55]. It is worth investigating in future studies which software performs the best in complicated agricultural environments, as the internal photogrammetry algorithms of the software have likely been customized.
Since all information in a reconstructed point cloud comes from RGB images, image quality is another vital factor, which impacts reconstructed orchard point cloud quality. As previously discussed, data collection timing, flight altitude, flight speed, camera resolution, and camera pitch angle can all affect image quality. Time of day mainly influences image brightness, and a proper brightness level is necessary for images to achieve good dynamic ranges. Flight altitude determines the tradeoff between image similarity and image detail. High flight altitudes would ensure successful reconstruction of coarse orchard point clouds with fewer blossom details. Low flight altitudes would allow cameras to observe apple flowers better; yet, reconstructed point cloud quality can suffer when significant perspective distortions are present between images. Flight speed determines the balance between image overlap, image blurriness, and flight duration. In general, high image quality and overlap should be prioritized over short data collection duration. Although it seems to be apparent that high camera resolution is advantageous for generating high-quality point clouds, it also takes longer to reconstruct point clouds using high-resolution images. During photogrammetric flight missions, UAVs usually fly in a grid pattern with 90° pitched cameras to acquire images with large frontal and side overlaps. In this case, to better observe the flowers growing on the sides of the apple tree rows, the flight mission parameters, including flight altitude, flight path, and camera pitch angle, were specifically chosen and designed to achieve dense blossom point cloud reconstruction. In future research, the quality of point clouds reconstructed using images captured at different times of day can be first examined to determine whether environmental brightness is an important consideration during data collection. Various image preprocessing techniques can be explored to improve image quality consistency. The optimal combination of flight altitude, flight speed, and camera pitch angle should be studied to generate complete, dense point cloud models of orchards. It would be an interesting research topic to compare point clouds reconstructed using image sets with various resolutions of the same orchard and investigate when additional image resolution would no longer help increase blossom density estimation accuracy.
As mentioned in Section 4.1, considering that different apple varieties may have different characteristics, such as canopy density and flower size, density, and distribution during their blooming periods, exploring optimal saturation and intensity thresholds and blossom grid filter sizes for individual apple varieties might help improve blossom density estimation accuracy. In the current study, there were insufficient data points to properly calibrate and validate such relationships for each apple variety. However, the preliminary analysis returned near perfect linear relationships with approximate R2s of 1 and RMSEs of 0 for Spur Red Delicious, Jonamac, and Jonagold using the limited data.
Since apple blossoms are a leading indicator for crop load, their pink form at first and full pink stages might be just as useful as their white form at first and full bloom stages. Although pink flower buds are much smaller and likely harder to reconstruct than white blossoms, predicting blossom count and density utilizing the number of downsampled pink points of an apple orchard point cloud is a potential research topic. Consequently, the impacts of pink color threshold and pink grid filter size on blossom density estimation will need to be investigated. As another potential extension of the current study, the proposed blossom density estimation methodology might also be applicable to other fruit trees with white flowers, such as peach, pear, and citrus, although optimal white color threshold and blossom grid filter size are likely different than those for apple.

5. Conclusions

The current study developed an original apple blossom density mapping algorithm utilizing RGB point cloud models reconstructed by photogrammetry software, and it explored the functionality of UAV-based sensing and data processing for orchard blossom management. The algorithm completes a streamlined, user-friendly, cost-effective, and efficient apple orchard blossom density monitoring workflow, from autonomous orchard image collection using UAV, photogrammetry-based point cloud reconstruction, to blossom density map generation using the developed algorithm. The utility of the proposed algorithm lies in objective blossom density assessment and visualized blossom thinning guidance, and the investigated optimal blossom downsampling grid filter size and white color threshold allowed apple blossom number to be estimated with satisfactory accuracy. While the preliminary field experiment demonstrated the algorithm’s capability in capturing blossom density variations of different apple varieties at different dates despite its simplicity, future studies are still needed to further explore the impacts of photogrammetry software, UAV flight mission design, and image quality on algorithm performance.

Author Contributions

Conceptualization, W.Y.; methodology, W.Y.; software, W.Y.; validation, W.Y.; formal analysis, W.Y.; investigation, W.Y. and W.H.; data curation, W.Y.; writing—original draft preparation, W.Y.; writing—review and editing, W.Y., P.H.H., W.H. and L.H.; visualization, W.Y.; funding acquisition, W.Y. All authors have read and agreed to the published version of the manuscript.

Funding

This material is based upon work supported by the National Institute of Food and Agriculture, U.S. Department of Agriculture, through the Northeast Sustainable Agriculture Research and Education program under subaward number GNE21-274. This research was partially supported in part by the United States Department of Agriculture (USDA)’s National Institute of Food and Agriculture (NIFA) Federal Appropriations under Project PEN04653 and Accession No. 1016510.

Data Availability Statement

The data presented in this study are available on request from the corresponding author. The proposed algorithm, which is under continuous development and validation, is available on request from the corresponding author.

Conflicts of Interest

The authors declare no conflict of interest.

Appendix A. Detailed Algorithm Breakdown

Appendix A.1. Sample Data Collection and Point Cloud Generation

For illustration purposes, a set of sample RGB images of a blooming apple orchard were captured using a DJI Matrice 200 V2 (Shenzhen, China) equipped with a DJI Zenmuse XT2 on 1 May 2021. The apple orchard was located at Russell E. Larson Agricultural Research Center, Pennsylvania Furnace, PA, USA (40.707918° N, 77.954370° W), which consisted of four tree rows and two apple cultivars, including Jonagold and Daybreak Fuji. The orchard dimension was roughly 25 m × 15 m. An autonomous UAV flight was conducted using DJI GS Pro with a preplanned flight path (Figure A1a). The flight altitude was set at 15 m, and the UAV speed was set at 1 m/s. Images of the apple orchard were captured every second with a 40° camera pitch, resulting in a roughly 90% overlap between consecutively collected images. The image dataset contained, in total, 274 images with a 4000 × 3000 resolution. PIX4Dmapper (Prilly, Switzerland), a commercial photogrammetry software, was used to reconstruct orchard point cloud model with the image dataset (Figure A1b). Each point within the point cloud contains location (easting, northing, elevation) and color (red, green, blue) information.

Appendix A.2. Terrain Map Generation

As apple orchards might be located in mountainous regions, terrain unevenness or elevation difference within orchards should be accounted for when the mapping algorithm needs to allow users to define the desired tree height regions. A terrain map that contains the elevation information within an orchard can help adjust an orchard point cloud model into one with perfectly flat ground.

Appendix A.2.1. Point Cloud Downsampling

A raw point cloud reconstructed by photogrammetry techniques typically contains a large number of points and can be extremely slow to process using average computers. For example, the sample apple orchard point cloud model (Figure A1b) consisted of more than 200 million points. Converting point clouds into maps is a process of losing information or discarding unnecessary information, as image pixel numbers are limited. Therefore, a downsampling procedure is necessary to improve the efficiency of the mapping algorithm. A grid-averaging approach is employed in the mapping algorithm, which is essentially the same as point cloud voxelization, where points within a moving grid filter with a predefined size are merged into one. Figure A2 shows an example of the sample point cloud being downsampled by a 1 m × 1 m × 1 m terrain grid filter. The terrain grid filter size is an important parameter, which determines the spatial resolutions of final terrain maps as well as final blossom density maps. For instance, a 1 m terrain grid filer size will lead to a 1 m per pixel spatial resolution in the maps. Generally, a large grid filter size is recommended for generating a smooth terrain map.
Horticulturae 09 00266 g0a1 550
Figure A1. The apple orchard where sample RGB image data were collected: (a) the flight path of the UAV flight mission; (b) the reconstructed apple orchard point cloud using the sample dataset.
Figure A1. The apple orchard where sample RGB image data were collected: (a) the flight path of the UAV flight mission; (b) the reconstructed apple orchard point cloud using the sample dataset.
Horticulturae 09 00266 g0a1

Appendix A.2.2. Height Map Generation

To generate a terrain map, ground surface objects with substantial heights, such as apple trees and wood posts, need to be identified and removed. The principle of identifying apple trees in the mapping algorithm rests on locating downsampled “unit regions” with in-unit maximum point elevation differences exceeding a predefined threshold. For example, in Figure A2b, each point represents a 1 m × 1 m downsampled unit region, which typically contains multiple points with identical eastings and northings but different elevations. Unit regions where apple trees are located generally have larger in-unit point elevation variations than those only containing ground points, since an apple tree can be as tall as a few meters, while the height of grasses on the ground is in the magnitude of tens of centimeters.
Given a downsampled orchard point cloud, a maximum height map, a minimum height map, and a height difference map are generated. The maximum and minimum height maps are created by a two-step process. First, “empty” height images or 2D arrays, whose dimensions are determined by the numbers of unique eastings and northings in the downsampled point cloud, are initialized. In the height images, the center of each row of pixels represents a constant northing, and the center of each column of pixels represents a constant easting. Second, the heights of the highest and lowest points within each unit region are mapped into the empty height images based on the eastings and northings accordingly (Figure A3a,b). The height difference map is calculated by subtracting the minimum height map from the maximum height map, which contains in-unit maximum point elevation difference information (Figure A3c). A tree mask was generated by thresholding the height difference map to remove the apple trees, and a 0.3 m threshold was adopted in the mapping algorithm (Figure A3d). Note the example minimum height map (Figure A3b) is very close to a terrain map because the apple tree canopies were not dense and low enough, such that the ground below the tree canopies could not be observed by the camera during data collection; hence, the ground points below the apple trees could still be reconstructed. The height difference map is necessary for removing ground surface objects that fully occlude the ground, such as a crate, which would appear as abnormally bright pixels in the minimum height map.
Horticulturae 09 00266 g0a2 550
Figure A2. A top-view example of the downsampling process for the raw orchard point cloud by a 1 m × 1 m × 1 m terrain grid filter: (a) before downsampling; (b) after downsampling.
Figure A2. A top-view example of the downsampling process for the raw orchard point cloud by a 1 m × 1 m × 1 m terrain grid filter: (a) before downsampling; (b) after downsampling.
Horticulturae 09 00266 g0a2
Horticulturae 09 00266 g0a3 550
Figure A3. Example height maps and tree mask derived from the downsampled orchard point cloud by a 0.1 m × 0.1 m × 0.1 m terrain grid filter: (a) maximum height map; (b) minimum height map; (c) height difference map; (d) tree mask.
Figure A3. Example height maps and tree mask derived from the downsampled orchard point cloud by a 0.1 m × 0.1 m × 0.1 m terrain grid filter: (a) maximum height map; (b) minimum height map; (c) height difference map; (d) tree mask.
Horticulturae 09 00266 g0a3

Appendix A.2.3. Terrain Map Interpolation and Smoothing

The base of a terrain map is the minimum height map (Figure A3b) masked by the tree mask (Figure A3d). To fill in the empty pixels, whose values are either removed by the tree mask or never assigned, a customized linear interpolation-based approach is used in the mapping algorithm. Denoting an empty pixel value as 0, a base terrain map in the interpolation process is interpolated horizontally and vertically, in total twice (Figure A4). Regardless of the interpolation direction, given a row or column of pixel values, an empty pixel or a series of connected empty pixels having non-zero neighbors on both sides are linearly interpolated (Figure A4). Since certain empty pixels can only be interpolated in one of the interpolation directions, the final interpolated terrain map is generated by “selectively averaging” the horizontally and vertically interpolated terrain maps. For the interpolated pixels that only exist in the horizontally or vertically interpolated terrain map, their values are preserved in the final interpolated terrain map. For the interpolated pixels that exist in both horizontally and vertically interpolated terrain maps, their values from the two maps are averaged and then used in the final interpolated terrain map (Figure A4). The overall interpolation process is repeated twice to create a terrain map without disconnected components and concave regions.
Horticulturae 09 00266 g0a4 550
Figure A4. Illustration of the terrain map interpolation process using a randomly generated image. Yellow regions highlight the empty pixels, which can potentially be interpolated.
Figure A4. Illustration of the terrain map interpolation process using a randomly generated image. Yellow regions highlight the empty pixels, which can potentially be interpolated.
Horticulturae 09 00266 g0a4
An optional final step of terrain map generation is smoothing, which can help reduce high frequency image noise if present. A standard image averaging filter is used in the mapping algorithm, while a Gaussian smoothing filter would work equally fine. The averaging filter has an odd size, which is user definable. When empty pixels are present within the filter, the filter ignores them and computes the average value of non-empty pixels. Considering the terrain map is created based on a downsampled point cloud having only a limited number of unique point elevation values, a smoothing operation is not essential for the mapping algorithm. Figure A5 shows a few examples of the final terrain maps created using the sample orchard point cloud downsampled by terrain grid filters with different sizes. Note a smaller grid filter size leads to more refined terrain maps.
Horticulturae 09 00266 g0a5 550
Figure A5. Example smoothed terrain maps of the sample orchard point cloud generated using different terrain grid filters. The maps were smoothed by a 3.3 m × 3.3 m averaging filter.
Figure A5. Example smoothed terrain maps of the sample orchard point cloud generated using different terrain grid filters. The maps were smoothed by a 3.3 m × 3.3 m averaging filter.
Horticulturae 09 00266 g0a5

Appendix A.3. Blossom Point Cloud Extraction and Downsampling

To generate a blossom density map, the identification of blossom points from a reconstructed orchard point cloud is essential. Utilizing the white color feature of apple blossoms, a color thresholding approach is used in the mapping algorithm to extract blossom point clouds. RGB values of the raw orchard point cloud (Figure A1b) are first converted into hue-saturation-intensity (HSI) values, which range from 0 to 1. Since white colors can be represented by either low-saturation values or high-intensity values, white blossom points are extracted by applying a threshold as the upper limit for the S channel and 1 minus the threshold as the lower limit for the I channel (Figure A6a). The optimal threshold level was explored in the field experiment, as explained in Section 3. As fallen petals on the ground and highly reflective surfaces also have white colors and will be extracted as part of the blossom point cloud, a simple height thresholding is utilized to eliminate the noisy points. Before that, the elevation values of the blossom point cloud need to be first adjusted by the terrain map, as apple trees can grow on hills with substantial slopes, and an elevation threshold will simply not work for such scenarios. Based on the easting and northing information, the terrain map pixel that contains a given blossom point can be located, and the elevation value of the pixel is then subtracted from the blossom point elevation value to calculate the point’s above-ground height. After all blossom point elevations are adjusted, a 0.3 m height threshold is applied to remove the noisy white points (Figure A6b).
Horticulturae 09 00266 g0a6 550
Figure A6. Blossom point cloud extraction from the sample orchard point cloud: (a) raw blossom point cloud extracted using white color thresholding; (b) height-adjusted, noise-removed, downsampled blossom point cloud.
Figure A6. Blossom point cloud extraction from the sample orchard point cloud: (a) raw blossom point cloud extracted using white color thresholding; (b) height-adjusted, noise-removed, downsampled blossom point cloud.
Horticulturae 09 00266 g0a6
The estimation of apple blossom count or density in the mapping algorithm is based on the presumption that downsampled blossom point numbers and actual blossom numbers in orchards are positively correlated. However, depending on object location, point clouds reconstructed by photogrammetry techniques can have vastly different local point cloud densities. For example, an apple blossom, which is at the top of a tree canopy and well observed by many images, may have a very dense reconstructed point cloud, while an apple blossom, which is hidden inside a tree canopy and can barely be seen in images, may only have a few points being reconstructed. The purpose of blossom point cloud downsampling is to unify point cloud density, so that regardless of the reconstructed point cloud quality, a downsampled white point always represents a certain volume of space, hence, a certain number of blossoms (Figure A7). The downsampling procedure for the blossom point cloud is identical to that for the raw reconstructed orchard point cloud (Figure A2), except for the grid filter size, and the optimal blossom grid filter size was investigated in the field experiment, as specified in Section 3.

Appendix A.4. User-Defined Tree Height Region

One of the mapping algorithm’s features is the ability to generate blossom density maps of a user-defined tree height region, which gives growers the freedom to precisely manage blossom thinning not only at different locations but also at different tree heights. This can be helpful when blossom densities at different apple tree heights vary substantially. After users input the maximum and minimum heights of the desired tree height regions, two height thresholds are applied to the height-adjusted downsampled blossom point cloud to remove blossom points outside the desired tree height regions.
Horticulturae 09 00266 g0a7 550
Figure A7. Graphic illustration showing densely and sparsely reconstructed point clouds of the same apple blossom being downsampled to the same number of points.
Figure A7. Graphic illustration showing densely and sparsely reconstructed point clouds of the same apple blossom being downsampled to the same number of points.
Horticulturae 09 00266 g0a7

Appendix A.5. Blossom Density Map Generation

Appendix A.5.1. Blossom Containing Volume Map Generation

Blossom density can be calculated as the ratio between the number of blossoms and the volume of space containing the blossoms. The number of blossoms in a unit region can be represented by the number of downsampled blossom points in the unit region, as mentioned above. The relationships between downsampled blossom point numbers and actual blossom counts were studied in the field experiment in Section 3. Considering trees in modern apple orchards are often planted in rows, which can be wall-shaped due to the adopted training systems, for simplicity, all unit regions belonging to the same apple tree row are assumed to have the same blossom containing height in the downsampled orchard point cloud. The blossom containing height range of a tree row is calculated as the difference between its maximum and minimum blossom point heights (Figure A8), and the blossom containing volume of a unit region is calculated as squared terrain grid filter size times the blossom containing height range (e.g., 0.1 m × 0.1 m × 1.5 m = 0.015 m3).
Horticulturae 09 00266 g0a8 550
Figure A8. Schematic diagram showing the calculation of blossom containing height range of a downsampled tree row point cloud from a side view.
Figure A8. Schematic diagram showing the calculation of blossom containing height range of a downsampled tree row point cloud from a side view.
Horticulturae 09 00266 g0a8
Similar to Appendix A.2.2, a blossom mask is generated by first initializing an empty image with the same size as the terrain map and changing a pixel’s value from 0 to 1 when a blossom point is found within the pixel’s unit region (Figure A9a). A morphological closing is then applied onto the blossom mask to form complete tree row regions containing blossom points, and the mask is also cleaned up by removing connected pixel components with small sizes (Figure A9b). When the terrain grid filter size is 0.1 m × 0.1 m × 0.1 m, a 100 connected pixel threshold is used in the mapping algorithm. Note for larger terrain grid filter sizes, a smaller connected pixel threshold should be used for mask cleaning, as each pixel represents a larger unit region. For each tree row, or each disconnected component in the blossom mask after mask cleaning, the maximum and minimum height values of the blossom points belonging to the tree row are used to calculate the blossom containing height range. The blossom containing volumes of different tree rows are conveniently calculated by multiplying their blossom containing height ranges with squared terrain grid filter size (Figure A9c). Generally, when apple cultivars have similar heights or user-specified tree height regions are narrow, no apparent difference will exist between blossom containing volumes of different tree rows in a blossom containing volume map.
Horticulturae 09 00266 g0a9 550
Figure A9. The blossom containing volume map generation process using the sample blossom point cloud downsampled by a 0.1 m × 0.1 m × 0.1 m blossom grid filter: (a) raw blossom mask; (b) closed, cleaned blossom mask; (c) blossom containing volume map.
Figure A9. The blossom containing volume map generation process using the sample blossom point cloud downsampled by a 0.1 m × 0.1 m × 0.1 m blossom grid filter: (a) raw blossom mask; (b) closed, cleaned blossom mask; (c) blossom containing volume map.
Horticulturae 09 00266 g0a9

Appendix A.5.2. Blossom Count and Density Calculation

Using the same principle as blossom mask generation, a blossom count map is generated by first initializing an empty image with the same size as the terrain map, and then for every downsampled blossom point that is found within a pixel’s unit region, increasing the pixel value by 1. After obtaining the total blossom point count within a pixel, the count is multiplied by an empirically determined factor to calculate the estimated blossom count, as mentioned in Appendix A.5.1. The optimal factor value was explored in the field experiment, as explained in Section 3. The final blossom density map is calculated as the ratio between the blossom count map and the blossom containing volume map (Figure A10).
Horticulturae 09 00266 g0a10 550
Figure A10. Example blossom density maps of the sample orchard point cloud at different tree height regions generated with a 0.1 m × 0.1 m × 0.1 m terrain grid filter.
Figure A10. Example blossom density maps of the sample orchard point cloud at different tree height regions generated with a 0.1 m × 0.1 m × 0.1 m terrain grid filter.
Horticulturae 09 00266 g0a10

References

  1. Spengler, R.N. Origins of the apple: The role of megafaunal mutualism in the domestication of Malus and rosaceous trees. Front. Plant Sci. 2019, 10, 617. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  2. USDA Foreign Agricultural Service. Fresh Apples, Grapes, and Pears: World Markets and Trade. 2022. Available online: https://www.fas.usda.gov/data/fresh-apples-grapes-and-pears-world-markets-and-trade (accessed on 12 January 2023).
  3. Crassweller, R.M.; Kime, L.F.; Harper, J.K. Apple Production. Agric. Altern. 2016, 1–12. [Google Scholar]
  4. Guitton, B.; Kelner, J.J.; Velasco, R.; Gardiner, S.E.; Chagné, D.; Costes, E. Genetic control of biennial bearing in apple. J. Exp. Bot. 2012, 63, 131–149. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  5. Pflanz, M.; Gebbers, R.; Zude, M. Influence of tree-adapted flower thinning on apple yield and fruit quality considering cultivars with different predisposition in fructification. Acta Hortic. 2016, 1130, 605–611. [Google Scholar] [CrossRef]
  6. Link, H. Significance of flower and fruit thinning on fruit quality. Plant Growth Regul. 2000, 31, 17–26. [Google Scholar] [CrossRef]
  7. Farjon, G.; Krikeb, O.; Hillel, A.B.; Alchanatis, V. Detection and counting of flowers on apple trees for better chemical thinning decisions. Precis. Agric. 2020, 21, 503–521. [Google Scholar] [CrossRef]
  8. Kolarič, J. Abscission of young apple fruits (Malus domestica Borkh): A review. Agricultura 2010, 7, 31–36. [Google Scholar]
  9. Apple Chemical Thinning. Available online: http://cpg.treefruit.wsu.edu/bioregulator-sprays/apple-chemical-thinning/ (accessed on 12 January 2023).
  10. Yoder, K.S.; Peck, G.M.; Combs, L.D.; Byers, R.E.; Smith, A.H. Using a pollen tube growth model to improve apple bloom thinning for organic production. Acta Hortic. 2013, 1001, 207–214. [Google Scholar] [CrossRef]
  11. Lakso, A.N.; Robinson, T.L.; Greene, D.W. Using an Apple Tree Carbohydrate Model to Understand Thinning Responses to Weather and Chemical Thinners. N. Y. State Hortic. Soc. 2002, 15, 16–19. [Google Scholar]
  12. Greene, D.W.; Lakso, A.N.; Robinson, T.L.; Schwallier, P. Development of a fruitlet growth model to predict thinner response on apples. HortScience 2013, 48, 584–587. [Google Scholar] [CrossRef]
  13. Yoder, K.; Yuan, R.; Combs, L.; Byers, R.; McFerson, J.; Schmidt, T. Effects of Temperature and the Combination of Liquid Lime Sulfur and Fish Oil on Pollen Germination, Pollen Tube Growth, and Fruit Set in Apples. HortScience 2009, 44, 1277–1283. [Google Scholar] [CrossRef] [Green Version]
  14. Robinson, T.L.; Lakso, A.N. Advances in Predicting Chemical Thinner Response of Apple Using a Carbon Balance Model. N. Y. Fruit Q. 2011, 19, 15–20. [Google Scholar] [CrossRef]
  15. Basak, A.; Juraś, I.; Białkowski, P.; Blanke, M.M.; Damerow, L. Efficacy of mechanical thinning of apple in Poland. Acta Hortic. 2016, 1138, 75–82. [Google Scholar] [CrossRef]
  16. Kon, T.M.; Schupp, J.R. Apple crop load management with special focus on early thinning strategies: A US perspective. In Horticultural Reviews; Wiley: Hoboken, NJ, USA, 2018; Volume 46, pp. 255–298. ISBN 9781119521082. Available online: https://onlinelibrary.wiley.com/doi/10.1002/9781119521082.ch6 (accessed on 12 January 2023).
  17. Schupp, J.R.; Auxt Baugher, T.; Miller, S.S.; Harsh, R.M.; Lesser, K.M. Mechanical thinning of peach and apple trees reduces labor input and increases fruit size. Horttechnology 2008, 18, 660–670. [Google Scholar] [CrossRef] [Green Version]
  18. Delavarpour, N.; Koparan, C.; Nowatzki, J.; Bajwa, S.; Sun, X. A Technical Study on UAV Characteristics for Precision Agriculture Applications and Associated Practical Challenges. Remote Sens. 2021, 13, 1204. [Google Scholar] [CrossRef]
  19. Nowak, B. Precision Agriculture: Where do We Stand? A Review of the Adoption of Precision Agriculture Technologies on Field Crops Farms in Developed Countries. Agric. Res. 2021, 10, 515–522. [Google Scholar] [CrossRef]
  20. Tsouros, D.C.; Triantafyllou, A.; Bibi, S.; Sarigannidis, P.G. Data acquisition and analysis methods in UAV-based applications for precision agriculture. In Proceedings of the 2019 15th International Conference on Distributed Computing in Sensor Systems (DCOSS), Santorini Island, Greece, 29–31 May 2019; pp. 377–384. [Google Scholar]
  21. Everaerts, J. The use of unmanned aerial vehicles (UAVs) for remote sensing and mapping. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sciendes 2014, 37, 1187–1191. [Google Scholar]
  22. Pajares, G. Overview and current status of remote sensing applications based on unmanned aerial vehicles (UAVs). Photogramm. Eng. Remote Sensing 2015, 81, 281–329. [Google Scholar] [CrossRef] [Green Version]
  23. Sun, Z.; Wang, X.; Wang, Z.; Yang, L.; Xie, Y.; Huang, Y. UAVs as remote sensing platforms in plant ecology: Review of applications and challenges. J. Plant Ecol. 2021, 14, 1003–1023. [Google Scholar] [CrossRef]
  24. Guo, Y.; Fu, Y.H.; Chen, S.; Robin Bryant, C.; Li, X.; Senthilnath, J.; Sun, H.; Wang, S.; Wu, Z.; de Beurs, K. Integrating spectral and textural information for identifying the tasseling date of summer maize using UAV based RGB images. Int. J. Appl. Earth Obs. Geoinf. 2021, 102, 102435. [Google Scholar] [CrossRef]
  25. Guo, Y.; Chen, S.; Li, X.; Cunha, M.; Jayavelu, S.; Cammarano, D.; Fu, Y.H. Machine Learning-Based Approaches for Predicting SPAD Values of Maize Using Multi-Spectral Images. Remote Sens. 2022, 14, 102435. [Google Scholar] [CrossRef]
  26. Tsouros, D.C.; Bibi, S.; Sarigiannidis, P.G. A review on UAV-based applications for precision agriculture. Information 2019, 10, 349. [Google Scholar] [CrossRef] [Green Version]
  27. Koenig, K.; Höfle, B.; Hämmerle, M.; Jarmer, T.; Siegmann, B.; Lilienthal, H. Comparative classification analysis of post-harvest growth detection from terrestrial LiDAR point clouds in precision agriculture. ISPRS J. Photogramm. Remote Sens. 2015, 104, 112–125. [Google Scholar] [CrossRef]
  28. Park, J.; Kim, H.; Tai, Y.-W.; Brown, M.S.; Kweon, I. High quality depth map upsampling for 3D-TOF cameras. In Proceedings of the 2011 International Conference on Computer Vision, Barcelona, Spain, 6–13 November 2011; pp. 1623–1630. [Google Scholar]
  29. Gee, A.; Prager, R.; Treece, G.; Berman, L. Engineering a freehand 3D ultrasound system. Pattern Recognit. Lett. 2003, 24, 757–777. [Google Scholar] [CrossRef]
  30. Armstrong, M.; Zisserman, A.; Beardsley, P. Euclidean Reconstruction from Uncalibrated Images. In Proceedings of the British Machine Vision Conference, York, UK, 13–16 September 1994; pp. 509–518. [Google Scholar]
  31. Pollefeys, M.; Koch, R.; Gool, L.V. Self-Calibration and Metric Reconstruction in spite of Varying and Unknown Intrinsic Camera Parameters. Int. J. Comput. Vis. 1999, 32, 7–25. [Google Scholar] [CrossRef]
  32. Seitz, S.M.; Curless, B.; Diebel, J.; Scharstein, D.; Szeliski, R. A comparison and evaluation of multi-view stereo reconstruction algorithms. Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. 2006, 1, 519–526. [Google Scholar]
  33. Johansen, K.; Raharjo, T.; McCabe, M.F. Using multi-spectral UAV imagery to extract tree crop structural properties and assess pruning effects. Remote Sens. 2018, 10, 854. [Google Scholar] [CrossRef] [Green Version]
  34. Mesas-Carrascosa, F.J.; Torres-Sánchez, J.; Clavero-Rumbao, I.; García-Ferrer, A.; Peña, J.M.; Borra-Serrano, I.; López-Granados, F. Assessing optimal flight parameters for generating accurate multispectral orthomosaicks by uav to support site-specific crop management. Remote Sens. 2015, 7, 12793–12814. [Google Scholar] [CrossRef] [Green Version]
  35. Dehkordi, R.H.; Burgeon, V.; Fouche, J.; Gomez, E.P.; Cornelis, J.T.; Nguyen, F.; Denis, A.; Meersmans, J. Using UAV collected RGB and multispectral images to evaluate winter wheat performance across a site characterized by century-old biochar patches in Belgium. Remote Sens. 2020, 12, 2504. [Google Scholar] [CrossRef]
  36. Taylor, J.E.; Charlton, D.; Yuńez-Naude, A. The End of Farm Labor Abundance. Appl. Econ. Perspect. Policy 2012, 34, 587–598. [Google Scholar] [CrossRef]
  37. Wang, X.A.; Tang, J.; Whitty, M. DeepPhenology: Estimation of apple flower phenology distributions based on deep learning. Comput. Electron. Agric. 2021, 185, 106123. [Google Scholar] [CrossRef]
  38. Dias, P.A.; Tabb, A.; Medeiros, H. Apple flower detection using deep convolutional networks. Comput. Ind. 2018, 99, 17–28. [Google Scholar] [CrossRef] [Green Version]
  39. Dias, P.A.; Tabb, A.; Medeiros, H. Multispecies fruit flower detection using a refined semantic segmentation network. IEEE Robot. Autom. Lett. 2018, 3, 3003–3010. [Google Scholar] [CrossRef] [Green Version]
  40. Wu, D.; Lv, S.; Jiang, M.; Song, H. Using channel pruning-based YOLO v4 deep learning algorithm for the real-time and accurate detection of apple flowers in natural environments. Comput. Electron. Agric. 2020, 178, 105742. [Google Scholar] [CrossRef]
  41. Wang, X.A.; Tang, J.; Whitty, M. Side-view apple flower mapping using edge-based fully convolutional networks for variable rate chemical thinning. Comput. Electron. Agric. 2020, 178, 105673. [Google Scholar] [CrossRef]
  42. Tubau Comas, A.; Valente, J.; Kooistra, L. Automatic apple tree blossom estimation from UAV RGB imagery. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 2019, 42, 631–635. [Google Scholar] [CrossRef] [Green Version]
  43. Tian, Y.; Yang, G.; Wang, Z.; Li, E.; Liang, Z. Instance segmentation of apple flowers using the improved mask R–CNN model. Biosyst. Eng. 2020, 193, 264–278. [Google Scholar] [CrossRef]
  44. Hočevar, M.; Širok, B.; Godeša, T.; Stopar, M. Flowering estimation in apple orchards by image analysis. Precis. Agric. 2014, 15, 466–478. [Google Scholar] [CrossRef]
  45. Braun, B.; Bulanon, D.M.; Colwell, J.; Stutz, A.; Stutz, J.; Nogales, C.; Hestand, T.; Verhage, P.; Tracht, T. A Fruit Yield Prediction Method Using Blossom Detection. In Proceedings of the ASABE 2018 Annual International Meeting, Detroit, MI, USA, 29 July 2018–1 August 2018; p. 1801542. [Google Scholar]
  46. Xiao, C.; Zheng, L.; Sun, H.; Zhang, Y.; Li, M. Estimation of the apple flowers based on aerial multispectral image. In Proceedings of the ASABE 2014 Annual International Meeting, Montreal, QC, Canada, 13–16 July 2014; p. 141912593. [Google Scholar]
  47. Yuan, W.; Choi, D. UAV-Based Heating Requirement Determination for Frost Management in Apple Orchard. Remote Sens. 2021, 13, 273. [Google Scholar] [CrossRef]
  48. Wang, C.; Liu, Y.; Zhang, Z.; Han, L.; Li, Y.; Zhang, H.; Wongsuk, S.; Li, Y.; Wu, X.; He, X. Spray performance evaluation of a six-rotor unmanned aerial vehicle sprayer for pesticide application using an orchard operation mode in apple orchards. Pest Manag. Sci. 2022, 78, 2449–2466. [Google Scholar] [CrossRef]
  49. Liu, Z.; Guo, P.; Liu, H.; Fan, P.; Zeng, P.; Liu, X.; Feng, C.; Wang, W.; Yang, F. Gradient boosting estimation of the leaf area index of apple orchards in uav remote sensing. Remote Sens. 2021, 13, 3263. [Google Scholar] [CrossRef]
  50. Dong, X.; Kim, W.Y.; Lee, K.H. Drone-Based Three-Dimensional Photogrammetry and Concave Hull by Slices Algorithm for Apple Tree Volume Mapping. J. Biosyst. Eng. 2021, 46, 474–484. [Google Scholar] [CrossRef]
  51. Valente, J.; Almeida, R.; Kooistra, L. A comprehensive study of the potential application of flying ethylene-sensitive sensors for ripeness detection in apple orchards. Sensors 2019, 19, 372. [Google Scholar] [CrossRef] [Green Version]
  52. Xiao, D.; Pan, Y.; Feng, J.; Yin, J.; Liu, Y.; He, L. Remote sensing detection algorithm for apple fire blight based on UAV multispectral image. Comput. Electron. Agric. 2022, 199, 107137. [Google Scholar] [CrossRef]
  53. Apolo-Apolo, O.E.; Pérez-Ruiz, M.; Martínez-Guanter, J.; Valente, J. A Cloud-Based Environment for Generating Yield Estimation Maps From Apple Orchards Using UAV Imagery and a Deep Learning Technique. Front. Plant Sci. 2020, 11, 1086. [Google Scholar] [CrossRef] [PubMed]
  54. Qureshi, A.H.; Alaloul, W.S.; Murtiyoso, A.; Saad, S.; Manzoor, B. Comparison of Photogrammetry Tools Considering Rebar Progress Recognition. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 2022, XLIII-B2-2, 141–146. [Google Scholar] [CrossRef]
  55. Maiwald, F.; Maas, H.G. An automatic workflow for orientation of historical images with large radiometric and geometric differences. Photogramm. Rec. 2021, 36, 77–103. [Google Scholar] [CrossRef]
Horticulturae 09 00266 g001 550
Figure 1. Schematic flowchart of the proposed apple orchard blossom density mapping algorithm.
Figure 1. Schematic flowchart of the proposed apple orchard blossom density mapping algorithm.
Horticulturae 09 00266 g001
Horticulturae 09 00266 g002 550
Figure 2. Illustration of manual apple blossom counting during the field experiment: (a) top view of the apple orchard showing the tarp strips, which divided each tree row into three sections; (b) blossom counting of one side of a tree row section.
Figure 2. Illustration of manual apple blossom counting during the field experiment: (a) top view of the apple orchard showing the tarp strips, which divided each tree row into three sections; (b) blossom counting of one side of a tree row section.
Horticulturae 09 00266 g002
Horticulturae 09 00266 g003 550
Figure 3. Examples of apple flowers that were excluded (first row) or included (second row) during the manual blossom counting: (a) an unopened blossom with pink petals; (b) a half-opened blossom with pinkish petals; (c) a wilted and shrunken blossom; (d) a half-opened blossom with white petals; (e) a fully opened white blossom; (f) a wilted blossom of substantial size.
Figure 3. Examples of apple flowers that were excluded (first row) or included (second row) during the manual blossom counting: (a) an unopened blossom with pink petals; (b) a half-opened blossom with pinkish petals; (c) a wilted and shrunken blossom; (d) a half-opened blossom with white petals; (e) a fully opened white blossom; (f) a wilted blossom of substantial size.
Horticulturae 09 00266 g003
Horticulturae 09 00266 g004 550
Figure 4. The apple orchard where the field experiment data collection was conducted: (a) the flight path of the UAV flight missions; (b) a reconstructed apple orchard point cloud using the set of RGB images collected on 3 May 2022 that shows the tarp strips.
Figure 4. The apple orchard where the field experiment data collection was conducted: (a) the flight path of the UAV flight missions; (b) a reconstructed apple orchard point cloud using the set of RGB images collected on 3 May 2022 that shows the tarp strips.
Horticulturae 09 00266 g004
Horticulturae 09 00266 g005 550
Figure 5. R2 and RMSE heatmaps of the 420 linear relationships between manual blossom counts and downsampled white blossom point numbers extracted from the 24 cropped tree row section point clouds using different white color thresholds and blossom grid filter sizes.
Figure 5. R2 and RMSE heatmaps of the 420 linear relationships between manual blossom counts and downsampled white blossom point numbers extracted from the 24 cropped tree row section point clouds using different white color thresholds and blossom grid filter sizes.
Horticulturae 09 00266 g005
Horticulturae 09 00266 g006 550
Figure 6. Field photos of apple blossoms at different developmental stages with different sizes: (a) small flowers with closed petals; (b) large flowers with fully opened petals; (c) small flowers with wilted petals.
Figure 6. Field photos of apple blossoms at different developmental stages with different sizes: (a) small flowers with closed petals; (b) large flowers with fully opened petals; (c) small flowers with wilted petals.
Horticulturae 09 00266 g006
Horticulturae 09 00266 g007 550
Figure 7. The best regression relationship achieved between downsampled blossom point number and manual blossom count using a 0.04 white color threshold and a 0.09 m × 0.09 m × 0.09 m blossom grid filter.
Figure 7. The best regression relationship achieved between downsampled blossom point number and manual blossom count using a 0.04 white color threshold and a 0.09 m × 0.09 m × 0.09 m blossom grid filter.
Horticulturae 09 00266 g007
Horticulturae 09 00266 g008 550
Figure 8. Blossom density maps of the field experiment apple orchard on multiple dates during the growing season of 2022 generated with a 0.3 m × 0.3 m × 0.3 m terrain grid filter.
Figure 8. Blossom density maps of the field experiment apple orchard on multiple dates during the growing season of 2022 generated with a 0.3 m × 0.3 m × 0.3 m terrain grid filter.
Horticulturae 09 00266 g008
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Yuan, W.; Hua, W.; Heinemann, P.H.; He, L. UAV Photogrammetry-Based Apple Orchard Blossom Density Estimation and Mapping. Horticulturae 2023, 9, 266. https://doi.org/10.3390/horticulturae9020266

AMA Style

Yuan W, Hua W, Heinemann PH, He L. UAV Photogrammetry-Based Apple Orchard Blossom Density Estimation and Mapping. Horticulturae. 2023; 9(2):266. https://doi.org/10.3390/horticulturae9020266

Chicago/Turabian Style

Yuan, Wenan, Weiyun Hua, Paul Heinz Heinemann, and Long He. 2023. "UAV Photogrammetry-Based Apple Orchard Blossom Density Estimation and Mapping" Horticulturae 9, no. 2: 266. https://doi.org/10.3390/horticulturae9020266

APA Style

Yuan, W., Hua, W., Heinemann, P. H., & He, L. (2023). UAV Photogrammetry-Based Apple Orchard Blossom Density Estimation and Mapping. Horticulturae, 9(2), 266. https://doi.org/10.3390/horticulturae9020266

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop