Next Article in Journal
Metabolic Profiles of Pomegranate Juices during Fruit Development and the Redirection of Flavonoid Metabolism
Next Article in Special Issue
Correlation of the Grapevine (Vitis vinifera L.) Leaf Chlorophyll Concentration with RGB Color Indices
Previous Article in Journal
Taxonomic Comparison, Antioxidant and Antibacterial Activities of Three Ebenus pinnata Ait. ecotypes (Fabaceae) from Algeria
Previous Article in Special Issue
In-Orchard Sizing of Mango Fruit: 2. Forward Estimation of Size at Harvest
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Horticulturae
Volume 9
Issue 8
10.3390/horticulturae9080880
horticulturae-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Evaluation of Computer Vision Systems and Applications to Estimate Trunk Cross-Sectional Area, Flower Cluster Number, Thinning Efficacy and Yield of Apple

by
Luis Gonzalez Nieto
1,*,
Anna Wallis
2,
Jon Clements
3,
Mario Miranda Sazo
4,
Craig Kahlke
4,
Thomas M. Kon
5 and
Terence L. Robinson
1
1
Horticulture Section, School of Integrative Plant Science, Cornell University, Geneva, NY 14456, USA
2
Extension Section, Michigan State University, Grand Rapids, MI 49503, USA
3
UMass Extension Section, University of Massachusetts, Belchertown, MA 01007, USA
4
Cornell Cooperative Extension, Lake Ontario Fruit Program, Albion, NY 14411, USA
5
Mountain Horticultural Crops Research and Extension Center, Department of Horticultural Sciences, North Carolina State University, Mills River, NC 28759, USA
*
Author to whom correspondence should be addressed.
Horticulturae 2023, 9(8), 880; https://doi.org/10.3390/horticulturae9080880
Submission received: 25 June 2023 / Revised: 25 July 2023 / Accepted: 26 July 2023 / Published: 3 August 2023
(This article belongs to the Special Issue Plant-Based, Proximal and Remote Sensing in Orchards and Vineyards — State of the Art, Challenges, Data Fusion and Integration)
Browse Figures
Review Reports Versions Notes

Abstract

:
Precision crop load management of apple requires counting fruiting structures at various times during the year to guide management decisions. The objective of the current study was to evaluate the accuracy of and compare different commercial computer vision systems and computer applications to estimate trunk cross-sectional area (TCSA), flower cluster number, thinning efficacy, and yield estimation. These studies evaluated two companies that offer different vision systems in a series of trials across 23 orchards in four states. Orchard Robotics uses a proprietary camera system, and Pometa (previously Farm Vision) uses a cell phone camera system. The cultivars used in the trials were ‘NY1’, ‘NY2’, ‘Empire’, ‘Granny Smith’, ‘Gala’, ‘Fuji’, and ‘Honeycrisp’. TCSA and flowering were evaluated with the Orchard Robotics camera in full rows. Flowering, fruit set, and yield estimation were evaluated with Pometa. Both systems were compared with manual measurements. Our results showed a positive linear correlation between the TCSA with the Orchard Robotics vision system and manual measurements, but the vision system underestimated the TCSA in comparison with the manual measurements (R2s between 0.5 and 0.79). Both vision systems showed a positive linear correlation between nubers of flowers and manual counts (R2s between 0.5 and 0.95). Thinning efficacy predictions (in June) were evaluated using the fruit growth rate model, by comparing manual measurements and the MaluSim computer app with the computer vision system of Pometa. Both systems showed accurate predictions when the numbers of fruits at harvest were lower than 200 fruit/tree, but our results suggest that, when the numbers of fruits at harvest were higher than 200 fruit/tree, both methods overestimated final fruit numbers per tree when compared with final fruit numbers at harvest (R2s 0.67 with both systems). Yield estimation was evaluated just before harvest (August) with the Pometa system. Yield estimation was accurate when fruit numbers were fewer than 75 fruit per tree, but, when the numbers of fruit at harvest were higher than 75 fruit per tree, the Pometa vision system underestimated the final yield (R2 = 0.67). Our results concluded that the Pometa system using a smartphone offered advantages such as low cost, quick access, simple operation, and accurate precision. The Orchard Robotics vision system with an advanced camera system provided more detailed and accurate information in terms of geo-referenced information for individual trees. Both vision systems evaluated are still in early development and have the potential to provide important information for orchard managers to improve crop load management decisions.

1. Introduction

Precision crop load management is the single most important yet difficult management strategy that determines the annual profitability of apple orchards [1]. Apple trees generally produce too many flower clusters and fruit for an optimum crop load [2]. Only about 3–10% of the initial fruit population should be carried to harvest to optimize crop value and promote annual bearing. When the fruit set is too high, low-quality fruit are produced and biennial bearing is induced [3]. There is a positive correlation between crop load and high fruit yield [4]. Crop load is a common parameter used in the industry that and can be defined simply as the number of fruits per tree or, more commonly, as the number of fruits per trunk cross-sectional area (fruit/TCSA) [5,6,7]. TCSA is often used to define the optimum target crop load for a tree or an orchard. Several studies have estimated that the optimal crop load in apple is between six and eight fruits/TCSA [1,8,9]. The traditional method of calculating TCSA is to measure the trunk circumference at 30 cm above the graft union with a tape measure or caliper and then calculate the cross-sectional area [10]. Recently, several studies have focused on using computer vision systems to estimate the TCSA [8,11,12].
The concepts of precision crop load management were developed to help growers manage crop load in a systematic manner to optimize fruit load and crop value [13]. The process involves precision pruning, precision chemical thinning, and precision hand thinning. Precision pruning is a process of the stepwise removal of flower buds to a pre-determined flower bud load before bud break in the spring. Precision chemical thinning is the sequential application of chemical thinning sprays, guided using computer models to adjust the rate and timing of chemical application. The models include the pollen tube growth model (PTGM) to time blossom thinning spray applications precisely [14,15,16,17]. This model requires a visual observation of the percentage of open blossoms. However, visual observation of open blooms is a subjective method.
The other two predictive models, the MaluSim model [18] and the fruit growth model, use precision crop load management [19]. The MaluSim model is a dynamic simulation model that uses weather variables to predict apple tree carbohydrate supply and demand [18], and is used to predict the rate of application and thinning efficacy. The fruit growth rate model is used to predict thinning efficacy after thinner application and determine whether additional applications are needed. It assumes faster growing fruitlets will persist and slower growing fruitlets will fall off. This model has been very useful in precision crop load management but its adoption has been limited due to the time requirement to carry out repeated manual measurements of fruitlet diameters.
Harvest time is another critical time when information estimates of fruit numbers and size are very valuable. Accurate forecasts of yield and fruit size can inform decisions for harvest labor needs and marketing plans for fruit growers and packing companies [20]. Presently, manual counting or visual assessments are used. These strategies can be difficult to execute during the harvest period and are imprecise.
Recent advancements in digital technology and computer vision are means to automate several steps in precision crop load management to manage pruning, flower and fruit thinning, and harvest more efficiently. The most important steps are automating the counting of flower buds in the dormant season, counting of flower clusters during bloom, measuring fruitlet diameters during the post-bloom thinning period, and counting final fruit numbers and measuring fruit size before harvest. This information is needed to inform management decisions [21]. Over the last several years, research has focused on evaluating several digital ag companies on computer vision tools to streamline the counting of apple buds, flowers, fruitlets, and fruit at harvest to guide human or robotic workers to optimize crop load while minimizing labor costs [22]. The objective of the current study was to evaluate the accuracy of several commercial computer vision systems and computer applications to estimate trunk cross-sectional area (TCSA), flower cluster number, thinning efficacy, and yield estimation.

2. Materials and Methods

2.1. Plant Material and Sites

Experiments were conducted in 2021, 2022, and 2023 in commercial apple (Malus domestica Borkh.) orchards in Massachusetts, Michigan, New York, and North Carolina (Figure 1 and Table 1, Table 2, Table 3 and Table 4). Mature and uniform apple trees at each location were used in all trials with one exception. Specifically, 3-year-old ‘Granny Smith’ apples were evaluated in one experiment in North Carolina (Table 1, Table 2, Table 3 and Table 4). The cultivars used in the trials were ‘NY1′, ‘NY2′, ‘Empire’, ‘Granny Smith’, ‘Gala’, ‘Fuji’, and ‘Honeycrisp’ (Table 1, Table 2, Table 3 and Table 4). Orchards were managed with standard commercial practices used in each region.

2.2. Trunk Cross-Sectional Area (TCSA)

The evaluations of TCSA were conducted in 2023 with the Orchard Robotics vison system and manual measurements in an orchard in Geneva, New York. Full rows of ‘Gala’ (1–90 trees, 2–30 trees), ‘Honeycrisp’ (90 trees), and ‘Fuji’ (30 trees) were evaluated (Table 1 and Figure 2). A tape measure was used for manual measurements of trunk circumference and converted to square centimeters (cm2) cross-sectional areas. The TCSA was measured at 30 cm above the graft union. The Orchard Robotics vision system takes a photo of each single tree and detects the visible portion of the trunk (between graft union and first branches), and then measures the trunk diameter in the middle of the section that it can detect and calculates the TCSA.
Table 1. Characteristics of the orchards used for TCSA estimates in 2023.
Table 1. Characteristics of the orchards used for TCSA estimates in 2023.
LocationCultivarYear PlantedRootstockSystemSpacing (m)
(Number Tree per Row)
New York‘Gala’-1 2006G.11Tall Spindle0.9 × 3.4 (97)
‘Honeycrisp’ 2006M.9Tall Spindle0.9 × 3.4 (97)
‘Gala’-2 2019G.41Tall Spindle0.9 × 3.4 (30)
‘Fuji’ 2019G.41Tall Spindle0.9 × 3.4 (30)

2.3. Blossom Counts

In 2023, the accuracy of two computer vision systems was evaluated in New York (Table 2). Manual and computer vision estimates of the number of flower clusters per tree were carried out at the pink-bud stage (BBCH 61-65). The Pometa vision system was evaluated in a commercial orchard on ‘Gala’ and ‘NY1’ (Table 2). Pometa cluster counting was a 2023 beta product. Five representative trees were selected per block, which were scanned using a video function on a compatible cellular phone (Apple iPhone 14 pro (Cupertino, CA 95014, USA)), according to the company’s directions (Figure 3). Two videos were taken of each tree (one on each side), with a total of ten videos. Videos were uploaded to the Pometa website for data processing. After 24 h, results of the processed videos were available and were downloaded.
The Orchard Robotics vision system was evaluated in a commercial orchard of ‘Gala’ and ‘Fuji’ (Table 2). One full row (130 trees) for each variety was scanned with the Orchard Robotics system, while 30 trees in each row were selected for manual counts. The Orchard Robotics vision system was evaluated using manufacturer recommendations. At full bloom, one side of the row (well-exposed, sunny side) was scanned with the camera system mounted on an orchard vehicle at 8 km/h. All scans were uploaded automatically from the cam to the Orchard Robotics website for processing (Figure 4). After 24 h, the results of the processed videos were available on the company’s website. However, it was necessary to record the manual counts at the same time as the evaluation of the representative 5 trees to calibrate the system.
Table 2. Characteristics of the orchards used to evaluate bloom intensity in 2023.
Table 2. Characteristics of the orchards used to evaluate bloom intensity in 2023.
LocationCultivarRootstockSystemSpacing (m)2023
Pometa
(n of Trees)		Orchard
Robotics
(n of Trees)
New York‘NY1’G.41Tall Spindle0.9 × 3.44
‘Gala’G.41Tall Spindle0.9 × 3.4630
‘Fuji’G.41Tall Spindle0.9 × 3.4 30

2.4. Thinning Efficacy

The trials were carried out in 13 commercial orchards in 2022 in New York, Massachusetts, Michigan, and North Carolina. ‘Gala’, ‘Fuji’, and ‘Honeycrisp’ cultivars were evaluated with Pometa scans and with an iPhone, and compared with the MaluSim app (manual measurements) (Table 3). At each orchard, five representative trees were selected and the total number of blossom clusters per each tree were counted. The final number of fruits was determined after natural fruit drop or at harvest. Commercial chemical thinning sprays were applied in each orchard between 6 and 8 mm fruit king diameter.
The MaluSim app was developed by Cornell University and includes the fruitlet growth rate model, apple carbohydrate model, and an irrigation model. The MaluSim fruit growth model is based in the traditional fruit growth model developed by Greene et al. [19]. The model began with the selection of five representative trees and the tagging of 15 clusters in each tree (75 clusters in total). After the thinning spray was applied, the fruit diameters of all fruitlets on the tagged cluster were measured at days 4 and 7 after application with a digital caliper. The total number of flower clusters per tree, number of flowers per cluster, target of fruit per tree, and all diameter measurements were entered into the MaluSim app, which then predicted the thinning efficacy and average number of fruits per tree (Figure 5).
The Pometa system of predicting thinning efficacy after a thinning spray is done by scanning the trees using an iPhone video recording of the tree at days 4 and 7 after thinner application. The same trees used for the MaluSim estimation of thinning efficacy were `used for the Pometa thinning efficacy estimation. The Pometa scanning was done using manufacturer recommendations and equipment: (smart phone with a stereo video camera, and enhanced GPS location identifier) (Figure 6). Two videos were captured of each tree (one video for each side of the tree and a total of ten videos). The videos were uploaded to the Pometa web site for processing. After 24 h the results of the processed videos were available at the company’s website (Figure 6).
Table 3. Characteristics of the orchards used to evaluate the fruit growth rate model (FGRM) in 2022.
Table 3. Characteristics of the orchards used to evaluate the fruit growth rate model (FGRM) in 2022.
LocationCultivarRootstockSystemSpacing (m)
Massachusetts‘Gala’M.9Tall Spindle0.9 × 3.7
‘Fuji’M.9Tall Spindle0.9 × 3.7
‘Honeycrisp’G.11Tall Spindle0.9 × 3.7
‘Gala’G.41Tall Spindle0.9 × 3.7
‘Honeycrisp’G.41Tall Spindle0.9 × 3.7
Michigan‘Honeycrisp’M.9Super Spindle0.6 × 3.4
‘Gala’G.11Super Spindle0.6 × 3.4
‘Fuji’M.9Vertical Axe1.5 × 3.7
‘Gala’M.9Tall Spindle1.2 × 3.7
New York‘Honeycrisp’B.9Tall Spindle1.5 × 3.5
‘Gala’G.11Tall Spindle0.9 × 3.4
‘Honeycrisp’M.9Tall Spindle0.9 × 3.4
North Carolina‘Gala’M.9Tall Spindle0.9 × 4

2.5. Yield Estimation

Yield estimation trials were carried out in 11 commercial orchards in Michigan, New York, and North Carolina in 2021 and 2022. ‘Evercrisp’, ‘Granny Smith’, ‘Gala’, ‘Fuji’, and ‘Honeycrisp’ cultivars were evaluated with the Pometa system of yield estimation using iPhone scans and then compared with manual counts of fruit numbers at harvest (Table 4). At each orchard, five uniform trees were selected for yield estimation with the Pometa system, and then manually counted at harvest. Two videos were taken for each tree evaluation (one video for each side of each tree, with a total of ten videos). All videos were uploaded to the Pometa website for processing of the video data. After 24 h, the results of the processed videos were available on the company’s website (Figure 7).
Table 4. Characteristics of the orchards used to evaluate yield estimation technology in 2021 and 2022.
Table 4. Characteristics of the orchards used to evaluate yield estimation technology in 2021 and 2022.
LocationCultivarRootstockSystemSpacing (m)
Michigan‘Honeycrisp’M.9Super Spindle0.6 × 3.4
‘Gala’G.11Super Spindle0.6 × 3.4
‘Fuji’M.9Vertical Axe1.5 × 3.7
‘Gala’M.9Tall Spindle1.2 × 3.7
New York‘Evercrisp’B.9Tall Spindle0.9 × 3.6
‘Fuji’B.9Tall Spindle0.6 × 3.4
‘Gala’G.11Tall Spindle0.9 × 3.4
‘Honeycrisp’M.9Tall Spindle0.9 × 3.4
North Carolina‘Gala’M.26Vertical Axis1.8 × 4.3
‘Honeycrisp’M.9Multi-leader1.8 × 4.3
‘Granny Smith’B.9Tall Spindle0.9 × 3.7

2.6. Statistical Analysis

Data were analyzed by linear regression analysis using SAS 9.4 (SAS Institute Inc., Cary, NC, USA, 2009) to evaluate the accuracy of the vision systems in estimating TCSA, bloom intensity, and thinning efficacy, compared with manual measurements. Quadratic regression was used to correlate yields estimation with vision system estimates and manual fruit counts at harvest. The bloom intensity statistical analysis was performed using ANOVA in SAS 9.4 (SAS Institute Inc., 2009, Cary, NC (27513), USA). Means were separated using Fisher’s LSD tests at p < 0.05.

3. Results

3.1. Trunk Cross-Sectional Area

A positive linear correlation was observed between the digital camera measurements of TCSA by the Orchard Robotics system and the manual measurements (Figure 8). The vision system underestimated the TCSA in comparison with the manual measurements in all four trials. The vision system calculated the TCSA using the average of all the visible trunk while the manual measurements were always measured at 30 cm above the graft union.

3.2. Blossom Counts

Both vision systems evaluated showed a positive linear correlation between estimated number of flowers per tree, measured with the vision systems and manual counts (Figure 9). Overall, both vision systems were accurate with their estimation of flower clusters per tree (1:1), except for ‘NY1′, in which Pometa underestimated the number of blossoms per tree (Figure 9).
When all the trees for each cultivar were averaged, the numbers of flower clusters, whether estimated by computer vision of counted manually, were very similar except for the cultivar ‘NY1′ (Figure 10). However, the regression analysis showed that the manually counted number of flower clusters of ‘NY1′ was related to the computer vision estimate, but not in a 1:1 relationship (Figure 9).

3.3. Thinning Efficacy

There were significant positive linear relationships between predicted fruit per tree early in the season (June, post-thinning) and final fruit harvested (Figure 11). Both methods (MaluSim and Pometa) overestimated the number of fruit at harvest over a range of different crop loads. When the final numbers of fruit at harvest were lower than 200 fruit/tree, the prediction of both systems was more accurate (near 1:1). That is, the numbers of fruit at harvest were similar to those predicted by each model 7 days after spraying the chemical thinning product. However, when the numbers of fruit at harvest were higher than 200 fruit/tree, both systems overestimated the numbers of fruit at harvest (Figure 11).
The predictions of final fruit numbers per tree of both models were correlated (Figure 12). However, when the numbers of estimated fruit were lower than 200 fruit/tree, the Pometa system overestimated more than the MaluSim app. On the other hand, when the numbers of predicted fruit were higher than 200 fruit/tree, both systems showed the same prediction of numbers of fruit.

3.4. Yield Estimation

There was a significant correlation between fruit yield estimation late in the season (August/September, immediately before harvest) with the Pometa vision system and final yield at harvest (Figure 13). The estimation was accurate (1:1) when the final fruit numbers were fewer than 75 fruit per tree. When the numbers of fruit at harvest were higher than 75 fruit per tree, the vision system underestimated the final yield, except in one plot that overestimated the yield. Moreover, as the final fruit numbers per tree increased, the error in the estimate was higher (Figure 13).

4. Discussion

Precision agriculture uses information obtained from temporal data to address variability within an orchard, with the objective of reducing the variability through variable management and increase the crop value [23]. As growers adopt digital technologies, it is critical to validate the precision of the new technologies [1,24].
The variation in tree size within an orchard makes establishing a uniform target number of fruit per tree difficult, since the target for each tree is different. TCSA is a metric that can be used to quantify this variability and can be used to estimate optimum tree crop load in terms of wood mass [11,25,26]. Computer vision systems to measure TSCA allow estimates of each tree’s optimum economic crop load. Our results show significant variation in TCSA (tree size) around the mean of 40 cm2 in older trees and 4 cm2 in young trees, similar to that shown by Serra, et al. [27]. Our results show that a positive linear correlation was observed between the vision system estimates of TCSA and the manual measurements, similar to the results of Wang et al. [11]. However, in our data, the vision system underestimated the TCSA in comparison with the manual measurements. Wang et al. [11] justified the differences between evaluations with a vision system and manual measurements because the camera sometimes failed to detect the target tree or because there were significant occlusions caused by leaves or overlaps between the foreground and background trunks, concurring with our results. Another reason for the differences between the manual measurements and the vision system is that the vision system calculated the TCSA using the average diameter of all the visible trunk and the manual measurements were always measured at 30 cm above the graft union. The value of using the TCSA as a measure of a tree’s capacity to carry fruit is likely very good when the trees are young [25]; however, when trees fully fill their allotted space and arrive at the period of maximum yield, around 10 years after plantation, the TCSA continues to increase, but the yield is constant [28]. Thus, the optimum crop load based on TCSA need to be adjusted down each year as the TCSA continues to increase but the productive capacity of the tree remains constant. With the proper adjustment, the TCSA could still quantify the variability in tree size in an orchard to allow differential crop loads based on the trees’ capacity.
Both of the vision systems showed a positive linear correlation between numbers of flower clusters at full bloom and manual counts. This is consistent with the results of two other systems, the Cartographer [29] and unmanned aerial vehicles [30]. The Orchard Robotics system consistently provided accurate counts when properly calibrated with manual counts to account for occluded flower clusters. The Pometa system showed variability in the results, with one block showing accurate counts and another with large differences between the manual flowers counted and those detected with the vision system. However, both blocks showed a high linear correlation, and it is possible that the overall error could be reduced with a correction factor based on manual counts of representative trees. It appears that both systems could be used to manage the flower thinning with precision. For example, Penzel, et al. [31] considered estimates of flower intensity as essential to manage mechanical flower thinning. In addition, these systems could be used to estimate the best timing to initiate the pollen tube growth model, developed by Yoder, Peck, Combs and Byers [16], and Peck, Combs, DeLong and Yoder [17]. The geo-referenced information on flower cluster density can be used to create accurate heat maps to guide variable smart sprayers to manage the variable rate application of chemical thinning agents to match chemical dosage with flower cluster density on each tree.
Greene et al. [19] developed a model (fruit growth rate model) to predict the efficiency of thinning and to inform decisions on the necessity of more thinning sprays. The model uses the growth rate of tagged and measured fruitlets to predict the percentage that will abscise. A mobile phone app (MaluSim) was developed by Francescato and Robinson (unpublished) to automate the calculations necessary to run this model, which improved the efficiency and accuracy of this method, although it still requires manual measurements of fruit diameters. The recommended method is to measure five representative trees × 15 clusters per tree × 5 fruitlets per cluster, for a total of 375 fruitlets. Manual measurements require about two hours of work per orchard on each date. At least two measurements are required to estimate fruit per tree. The Pometa vision system uses the same concept of the fruit growth rate model but uses computer vision to measure the fruit diameters. This system utilizes a cell phone camera to capture videos of five representative trees. The videos are processed by a proprietary algorithm to determine which fruitlets will abscise. The time required to carry out all measurements is about 10 min per block. Ours results suggest that both systems (MaluSim and Pometa) showed accurate predictions of the numbers of fruit at harvest when the final fruit numbers were lower than 200 fruit/tree, concurring with Penzel, et al. [32], Costa, et al. [33], and McArtney, et al. [34]. However, our results suggest that, when the numbers of fruit at harvest were higher than 200 fruit/tree, both the MaluSim and the Pometa systems overestimated the numbers of fruit at harvest, which concurs with the results of Rufato, et al. [35]. The overestimation of fruit numbers per tree is likely due to the higher natural fruit drop after the chemical thinning window has passed [36]. This is typically attributed to a carbohydrate deficit in June, resulting in “June drop”. Our results also suggest that, when the numbers of fruit are higher, they consume greater amounts of carbohydrates and, as a consequence, the trees experience more frequent carbohydrate deficits, resulting in higher natural fruit abscission than in trees with a lower fruit load. Despite the overestimation of final fruit numbers when the crop load is high, our results show that both the manual model (MaluSim) and the computer vision system of Pometa are correlated, and the estimates with the computer vison system are very close to the estimates with manual measurements. Thus, the vision system with a cell phone camera will likely become a preferred method to estimate thinning efficacy in the future since it requires less time and provides easy interpretation of the results.
Estimation of harvest provides important information to apple orchard managers, storage operators, and fruit marketers. For automated harvests, yield maps will be required. Apple yield estimation with a smartphone offers the advantages of low cost, quick access, and simple operation [37]. Our results suggest that the use of smartphones to carry out yield estimation produced accurate results (1:1), until the numbers of fruit exceeded 75 fruit per tree. Similar results were shown by Wang, et al. [38] with a Nikon D300s camera. However, when the numbers of fruit at harvest were higher than 75 fruit per tree, the Pometa vision system underestimated the final yield because there was significant fruit occlusion caused by foliage. Scalisi, et al. [29] showed good correlations between predictions with the Cartographer system and manual measurements as long as the maximum fruit number per tree was 80 fruit, which is similar to our results. Penzel, et al. [39], working with Lidar, showed better predictions with a high range of numbers of fruit per tree. Currently, the Pometa company is working on an occlusion model to improve the yield estimation with tree forms that have higher occlusion of fruit.

5. Conclusions

The use of the vision systems to estimate TCSA, bloom intensity, thinning efficiency, and yield generated accurate predictions and usable data for the growers. The vision systems could measure the TSCA for each tree to estimate the optimum fruit numbers for each tree. However, the vision system underestimated the TCSA in comparison with the manual measurements because the measurements with the vision system were carried out in a different place on the trunk. Both vision systems evaluated could estimate the numbers of flower clusters per tree accurately. This information when geo-referenced with each tree could allow precise management of flower-thinning spray applications. The fruit growth rate model using the MaluSim app and the Pometa system showed accurate predictions of thinning efficacy when the crop load was lower than 200 fruit/tree. However, our results suggest that, when the numbers of fruit at harvest were higher than 200 fruit/tree, both the MaluSim and the Pometa systems predicted higher numbers of fruit at harvest than were actually harvested. The vision system of Pometa with cell phones needed less time than the MaluSim method and provided easy interpretation of the results. The yield estimation with Pometa was accurate up to a crop load of 75 fruit per tree. However, when the numbers of fruit at harvest were higher than 75 fruit per tree, the Pometa vision system underestimated the final yield because there was significant fruit occlusion caused by foliage. Currently, Pometa is working with a new occlusion model to improve the yield estimation with tree forms with high occlusion of fruit.
Overall, precision crop load management with smartphone-bases systems offers advantages such as low cost, quick access, and simple operation. However, the vision systems with special cameras and geo-referencing of each tree offer tree-specific information that will allow precise crop load management of each tree.

Author Contributions

Conceptualization, L.G.N., T.L.R., A.W., J.C., M.M.S., C.K. and T.M.K.; methodology, L.G.N., T.L.R., A.W., J.C., M.M.S., C.K. and T.M.K.; software, L.G.N.; validation, L.G.N., T.L.R., A.W., J.C., M.M.S., C.K. and T.M.K.; formal analysis, L.G.N., A.W. and J.C.; investigation, L.G.N., T.L.R., A.W., J.C., M.M.S., C.K. and T.M.K.; resources, L.G.N., T.L.R., A.W., J.C., M.M.S., C.K. and T.M.K.; data curation, L.G.N., T.L.R., A.W., J.C., M.M.S., C.K. and T.M.K.; writing—original draft preparation, L.G.N.; writing—review and editing, L.G.N., T.L.R., A.W., C.K. and T.M.K.; visualization, L.G.N., T.L.R., A.W., J.C., M.M.S., C.K. and T.M.K.; supervision, T.L.R.; project administration T.L.R.; funding acquisition, T.L.R.; All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by the Specialty Crop Research Initiative project “Precision Crop Load Management of Apples” (NYG-632521).

Data Availability Statement

Not applicable.

Acknowledgments

This work was supported by the USDA National Institute of Food and Agriculture—Specialty Crop Research Initiative project “Precision Crop Load Management of Apples” (NYG-632521). We thank the Orchard Robotic (https://www.orchard-robotics.com) and Pometa (https://pometa.io) companies.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Robinson, T.L.; Gonzalez, L.; Cheng, L.; Ziang, Y.; Peck, G.; Arnoldussen, B.; Gomez, M.; Guerra, M.; Miranda Sazo, M.; Kahlke, C.; et al. Studies in precision crop load management of apple. Acta Hortic. 2023, 219–226. [Google Scholar] [CrossRef]
  2. Gonzalez, L.; Torres, E.; Àvila, G.; Bonany, J.; Alegre, S.; Carbó, J.; Martín, B.; Recasens, I.; Asin, L. Evaluation of chemical fruit thinning efficiency using Brevis® (Metamitron) on apple trees (‘Gala’) under Spanish conditions. Sci. Hortic. 2020, 261, 109003. [Google Scholar] [CrossRef]
  3. Gonzalez, L.; Torres, E.; Àvila, G.; Carbó, J.; Bonany, J.; Alegre, S.; Asin, L. Effect of thinning with metamitron, NAA, BA and naphthenic acids on apple (Malus domestica) trees. Plant Growth Regul. 2023. [Google Scholar] [CrossRef]
  4. Gonzalez, L.; Francescatto, P.; Lordan, J.; Robinson, T.L. Rate and timing of metamitron affect thinning efficacy of ‘Gala’ apple trees under American northeast conditions. Acta Hortic. 2022, 1344, 55–64. [Google Scholar] [CrossRef]
  5. Racskó, J. Crop load, fruit thinning and their effects on fruit quality of apple (Malus domestica Borkh.). Acta Agrar. Debreceniensis 2006, 29–35. [Google Scholar] [CrossRef]
  6. Wünsche, J.N.; Greer, D.H.; Laing, W.A.; Palmer, J.W. Physiological and biochemical leaf and tree responses to crop load in apple. Tree Physiol. 2005, 25, 1253–1263. [Google Scholar] [CrossRef]
  7. Verma, P.; Sharma, S.; Sharma, N.; Chauhan, N. Review on crop load management in apple (Malus × domestica Borkh.). J. Hortic. Sci. Biotechnol. 2023, 98, 299–321. [Google Scholar] [CrossRef]
  8. Ahmed, D.; Sapkota, R.; Churuvija, M.; Karkee, M. Machine Vision-Based Crop-Load Estimation Using YOLOv8. arXiv 2023, arXiv:2304.13282. [Google Scholar]
  9. Anthony, B.; Serra, S.; Musacchi, S. Optimizing Crop Load for New Apple Cultivar: “WA38”. Agronomy 2019, 9, 107. [Google Scholar] [CrossRef] [Green Version]
  10. Jung, S.-K.; Ryu, H.-U.; Gu, M.; Choi, H.-S.; Ryu, J.-H.; Cho, Y.-K. Growth and fruit production in young ‘Fuji’apple trees as affected by time of downward bending of branches. J. Hortic. Sci. Biotechnol. 2012, 87, 519–523. [Google Scholar] [CrossRef]
  11. Wang, T.; Sankari, P.; Brown, J.; Paudel, A.; He, L.; Karkee, M.; Thompson, A.; Grimm, C.; Davidson, J.; Todorovic, S. Automatic Estimation of Trunk cross Sectional Area Using Deep Learning; Collaborative Robotics & Intelligent Systems Institute, Oregon State University: Corvallis, OR, USA, 2023. [Google Scholar] [CrossRef]
  12. Bargoti, S.; Underwood, J.P.; Nieto, J.I.; Sukkarieh, S. A pipeline for trunk localisation using LiDAR in trellis structured orchards. In Field and Service Robotics, Proceedings of the 9th International Conference on Field and Service Robotics FSR, Brisbane, Australia, 9–11 December 2013; Springer: Berlin/Heidelberg, Germany, 2015; pp. 455–468. [Google Scholar]
  13. Robinson, T.; Lakso, A.; Greene, D.; Hoying, S. Precision crop load management. N. Y. Fruit Q. 2013, 21, 445–468. [Google Scholar]
  14. Allen, W.C.; Kon, T.; Sherif, S.M. Evaluation of Blossom Thinning Spray Timing Strategies in Apple. Horticulturae 2021, 7, 308. [Google Scholar] [CrossRef]
  15. Dennis, F.G. The history of fruit thinning. Plant Growth Regul. 2000, 31, 1–16. [Google Scholar] [CrossRef]
  16. Yoder, K.; Peck, G.; Combs, L.D.; Byers, R.E. Using a Pollen Tube Growth Model to Improve Apple Bloom Thinning for Organic Production. Acta Hortic. 2013, 1001, 207–214. [Google Scholar] [CrossRef]
  17. Peck, G.M.; Combs, L.D.; DeLong, C.; Yoder, K.S. Precision apple flower thinning using organically approved chemicals. Acta Hortic. 2016, 1137, 47–52. [Google Scholar] [CrossRef]
  18. Lakso, A.; Robinson, T. Decision support for apple thinning based on carbon balance modeling. Acta Hortic. 2015, 1068, 235–242. [Google Scholar] [CrossRef]
  19. Greene, D.W.; Lakso, A.N.; Robinson, T.L.; Schwallier, P. Development of a Fruitlet Growth Model to Predict Thinner Response on Apples. Hortscience 2013, 48, 584–587. [Google Scholar] [CrossRef]
  20. Janowski, A.; Kaźmierczak, R.; Kowalczyk, C.; Szulwic, J. Detecting Apples in the Wild: Potential for Harvest Quantity Estimation. Sustainability 2021, 13, 8054. [Google Scholar] [CrossRef]
  21. Mavridou, E.; Vrochidou, E.; Papakostas, G.; Pachidis, T.; Kaburlasos, V. Machine Vision Systems in Precision Agriculture for Crop Farming. J. Imaging 2019, 5, 89. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  22. Wallis, A.; Clements, J.; Sazo, M.M.; Kahlke, C.; Lewis, K.; Kon, T.; Gonzalez, L.; Jiang, Y.; Robinson, T. Digital Technologies for Precision Apple Crop Load Management (PACMAN) Part I: Experiences with Tools for Predicting Fruit Set Based on the Fruit Growth Rate Model. N. Y. Fruit Q. 2023, 31, 8–13. [Google Scholar]
  23. Blanco, V.; Willsea, N.; Campbell, T.; Howe, O.; Kalcsits, L. Combining thermal imaging and soil water content sensors to assess tree water status in pear trees. Front. Plant Sci. 2023, 14, 1197437. [Google Scholar] [CrossRef]
  24. Gonzalez Nieto, L.; Huber, A.; Gao, R.; Biasuz, E.C.; Cheng, L.; Stroock, A.D.; Lakso, A.N.; Robinson, T.L. Trunk Water Potential Measured with Microtensiometers for Managing Water Stress in “Gala” Apple Trees. Plants 2023, 12, 1912. [Google Scholar] [CrossRef] [PubMed]
  25. Lombard, P.; Callan, N.W.; Dennis, F.; Looney, N.; Martin, G.; Renquist, A.; Mielke, E. Towards a standardized nomenclature, procedures, values, and units in determining fruit and nut tree yield performance. Hortscience 1988, 23, 813–817. [Google Scholar] [CrossRef]
  26. Schupp, J.R.; Winzeler, H.E.; Kon, T.M.; Marini, R.P.; Baugher, T.A.; Kime, L.F.; Schupp, M.A. A method for quantifying whole-tree pruning severity in mature tall spindle apple plantings. Hortscience 2017, 52, 1233–1240. [Google Scholar] [CrossRef] [Green Version]
  27. Serra, S.; Leisso, R.; Giordani, L.; Kalcsits, L.; Musacchi, S. Crop load influences fruit quality, nutritional balance, and return bloom in ‘Honeycrisp’apple. Hortscience 2016, 51, 236–244. [Google Scholar] [CrossRef]
  28. Lepsis, J.; Blanke, M. The trunk cross-section area as a basis for fruit yield modelling in intensive apple orchards. Acta Hortic. 2004, 707, 231–235. [Google Scholar] [CrossRef]
  29. Scalisi, A.; McClymont, L.; Underwood, J.; Morton, P.; Scheding, S.; Goodwin, I. Reliability of a commercial platform for estimating flower cluster and fruit number, yield, tree geometry and light interception in apple trees under different rootstocks and row orientations. Comput. Electron. Agric. 2021, 191, 106519. [Google Scholar] [CrossRef]
  30. Yuan, W.; Hua, W.; Heinemann, P.H.; He, L. UAV Photogrammetry-Based Apple Orchard Blossom Density Estimation and Mapping. Horticulturae 2023, 9, 266. [Google Scholar] [CrossRef]
  31. Penzel, M.; Pflanz, M.; Gebbers, R.; Zude-Sasse, M. Tree-adapted mechanical flower thinning prevents yield loss caused by over-thinning of trees with low flower set in apple. Eur. J. Hortic. Sci. 2021, 86, 88–98. [Google Scholar] [CrossRef]
  32. Penzel, M.; Kröling, C. Thinning efficacy of metamitron on young “RoHo 3615” (Evelina®) apple. Sci. Hortic. 2020, 272, 109586. [Google Scholar] [CrossRef]
  33. Costa, G.; Botton, A.; Vizzotto, G. Fruit thinning: Advances and trends. Hortic. Rev. 2018, 46, 185–226. [Google Scholar] [CrossRef]
  34. McArtney, S.; Obermiller, J. Evaluation of a model to predict the response of “Gala” apples to chemical thinners. Acta Hortic. 2010, 884, 581–586. [Google Scholar] [CrossRef]
  35. Rufato, A.; Robinson, T.; Rufato, L. Precision thinning of “Royal Gala” apple trees using the fruit growth model. Acta Hortic. 2017, 1177, 399–404. [Google Scholar] [CrossRef] [Green Version]
  36. Lordan, J.; Reginato, G.H.; Lakso, A.N.; Francescatto, P.; Robinson, T.L. Natural fruitlet abscission as related to apple tree carbon balance estimated with the MaluSim model. Sci. Hortic. 2019, 247, 296–309. [Google Scholar] [CrossRef]
  37. Qian, J.; Xing, B.; Wu, X.; Chen, M.; Wang, Y.a. A smartphone-based apple yield estimation application using imaging features and the ANN method in mature period. Sci. Agric. 2018, 75, 273–280. [Google Scholar] [CrossRef]
  38. Wang, Q.; Nuske, S.; Bergerman, M.; Singh, S. Automated Crop Yield Estimation for Apple Orchards. In Experimental Robotics, Proceedings of the the 13th International Symposium on Experimental Robotics, Québec City, QC, Canada, 18–21 June 2012; Springer: Berlin/Heidelberg, Germany, 2013; pp. 745–758. [Google Scholar] [CrossRef]
  39. Penzel, M.; Tsoulias, N. Annual shoot growth on apple trees with variable canopy leaf area and crop load in response to LiDAR scanned leaf area to fruit ratio. Int. Agrophysics 2022, 36, 173–180. [Google Scholar] [CrossRef]
Horticulturae 09 00880 g001
Figure 1. Locations in the Eastern USA of the experiments conducted on digital vision systems in 2021, 2022, and 2023. Geolocation Massachusetts (42°15′13.3″ N 72°21′35.3″ W), Michigan (42°58′15.1″ N 85°40′17.6″ W), New York (42°52′39.9″ N 77°00′25.9″ W) and North Carolina (35°25′47.9″ N 82°33′29.9″ W).
Figure 1. Locations in the Eastern USA of the experiments conducted on digital vision systems in 2021, 2022, and 2023. Geolocation Massachusetts (42°15′13.3″ N 72°21′35.3″ W), Michigan (42°58′15.1″ N 85°40′17.6″ W), New York (42°52′39.9″ N 77°00′25.9″ W) and North Carolina (35°25′47.9″ N 82°33′29.9″ W).
Horticulturae 09 00880 g001
Horticulturae 09 00880 g002
Figure 2. Trunk detections and measurements of TCSA with the Orchard Robotics vision system. Picture (A) is of two young trees (5 years old) and picture (B) is of two older trees (17 years old).
Figure 2. Trunk detections and measurements of TCSA with the Orchard Robotics vision system. Picture (A) is of two young trees (5 years old) and picture (B) is of two older trees (17 years old).
Horticulturae 09 00880 g002
Horticulturae 09 00880 g003
Figure 3. Flower clusters identified with the Pometa vision system and information derived from the cluster counts. Blue circles illustrate where flowers were detected.
Figure 3. Flower clusters identified with the Pometa vision system and information derived from the cluster counts. Blue circles illustrate where flowers were detected.
Horticulturae 09 00880 g003
Horticulturae 09 00880 g004
Figure 4. Flower clusters identified with the Orchard Robotics vision system and information derived from the cluster counts. Red squares illustrate where flower clusters were detected.
Figure 4. Flower clusters identified with the Orchard Robotics vision system and information derived from the cluster counts. Red squares illustrate where flower clusters were detected.
Horticulturae 09 00880 g004
Horticulturae 09 00880 g005
Figure 5. Flower set estimation with the MaluSim app at one location in New York in 2022.
Figure 5. Flower set estimation with the MaluSim app at one location in New York in 2022.
Horticulturae 09 00880 g005
Horticulturae 09 00880 g006
Figure 6. Fruitlets detected with the Pometa vision system and information derived from the scans. Blue dots on the tree in the photograph indicate detected fruitlets. The dashboard shows the details of each scan, including number of fruitlets detected, population distribution, and the adjusted number of fruitlets expected to persist (blue dots in graph) or abscise (orange).
Figure 6. Fruitlets detected with the Pometa vision system and information derived from the scans. Blue dots on the tree in the photograph indicate detected fruitlets. The dashboard shows the details of each scan, including number of fruitlets detected, population distribution, and the adjusted number of fruitlets expected to persist (blue dots in graph) or abscise (orange).
Horticulturae 09 00880 g006
Horticulturae 09 00880 g007
Figure 7. Fruit detected with the Pometa vision system and information derived from the scans. Blue circles are fruit detected.
Figure 7. Fruit detected with the Pometa vision system and information derived from the scans. Blue circles are fruit detected.
Horticulturae 09 00880 g007
Horticulturae 09 00880 g008
Figure 8. Relationships between trunk cross-sectional area (cm2) evaluated by the Orchard Robotics vision system and manual measurements in 2023. ‘Gala’-1 and ‘Honeycrisp’ were older trees (17 years old), and ‘Gala’-2 and Fuji were young trees (4 years old). Each symbol represents 1 tree. All linear correlations were significant at p < 0.001.
Figure 8. Relationships between trunk cross-sectional area (cm2) evaluated by the Orchard Robotics vision system and manual measurements in 2023. ‘Gala’-1 and ‘Honeycrisp’ were older trees (17 years old), and ‘Gala’-2 and Fuji were young trees (4 years old). Each symbol represents 1 tree. All linear correlations were significant at p < 0.001.
Horticulturae 09 00880 g008
Horticulturae 09 00880 g009
Figure 9. Relationships between estimated flower clusters per tree evaluated with the Orchard Robotics and Pometa visions system and manual counts in 2023. Each symbol represents 1 tree.
Figure 9. Relationships between estimated flower clusters per tree evaluated with the Orchard Robotics and Pometa visions system and manual counts in 2023. Each symbol represents 1 tree.
Horticulturae 09 00880 g009
Horticulturae 09 00880 g010
Figure 10. Average flower clusters per tree of ‘Gala’, ‘Fuji’, and ‘NY1′, estimated with the computer vision systems of Orchard Robotics and Pometa compared with manual counts in the same orchards (Figure 9). Different letters denote significant differences (Fisher’s LSD tests at p < 0.05).
Figure 10. Average flower clusters per tree of ‘Gala’, ‘Fuji’, and ‘NY1′, estimated with the computer vision systems of Orchard Robotics and Pometa compared with manual counts in the same orchards (Figure 9). Different letters denote significant differences (Fisher’s LSD tests at p < 0.05).
Horticulturae 09 00880 g010
Horticulturae 09 00880 g011
Figure 11. Relationships between the actual fruit numbers per tree at harvest in September 2022 and the predicted fruit numbers per tree, using the growth model with the MaluSim app or the Pometa computer vision system 7 days after applying a chemical thinning spray in May 2022. Each symbol represents 1 orchard. All linear correlations were significant at p < 0.01.
Figure 11. Relationships between the actual fruit numbers per tree at harvest in September 2022 and the predicted fruit numbers per tree, using the growth model with the MaluSim app or the Pometa computer vision system 7 days after applying a chemical thinning spray in May 2022. Each symbol represents 1 orchard. All linear correlations were significant at p < 0.01.
Horticulturae 09 00880 g011
Horticulturae 09 00880 g012
Figure 12. Relationship between predicted numbers of fruit per tree with the MaluSim app and the Pometa system in 2022. Each data point represents 1 orchard.
Figure 12. Relationship between predicted numbers of fruit per tree with the MaluSim app and the Pometa system in 2022. Each data point represents 1 orchard.
Horticulturae 09 00880 g012
Horticulturae 09 00880 g013
Figure 13. Relationship between estimated fruit per tree with the Pometa vision system and actual fruit harvested in 2022. The dotted line represent a linear correlation and the script line is a quadratic regression. Each symbol represents 1 orchard.
Figure 13. Relationship between estimated fruit per tree with the Pometa vision system and actual fruit harvested in 2022. The dotted line represent a linear correlation and the script line is a quadratic regression. Each symbol represents 1 orchard.
Horticulturae 09 00880 g013
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Gonzalez Nieto, L.; Wallis, A.; Clements, J.; Miranda Sazo, M.; Kahlke, C.; Kon, T.M.; Robinson, T.L. Evaluation of Computer Vision Systems and Applications to Estimate Trunk Cross-Sectional Area, Flower Cluster Number, Thinning Efficacy and Yield of Apple. Horticulturae 2023, 9, 880. https://doi.org/10.3390/horticulturae9080880

AMA Style

Gonzalez Nieto L, Wallis A, Clements J, Miranda Sazo M, Kahlke C, Kon TM, Robinson TL. Evaluation of Computer Vision Systems and Applications to Estimate Trunk Cross-Sectional Area, Flower Cluster Number, Thinning Efficacy and Yield of Apple. Horticulturae. 2023; 9(8):880. https://doi.org/10.3390/horticulturae9080880

Chicago/Turabian Style

Gonzalez Nieto, Luis, Anna Wallis, Jon Clements, Mario Miranda Sazo, Craig Kahlke, Thomas M. Kon, and Terence L. Robinson. 2023. "Evaluation of Computer Vision Systems and Applications to Estimate Trunk Cross-Sectional Area, Flower Cluster Number, Thinning Efficacy and Yield of Apple" Horticulturae 9, no. 8: 880. https://doi.org/10.3390/horticulturae9080880

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop