Flowering Intensity Estimation Using Computer Vision

Abstract

Flowering intensity is an important parameter to predict and control fruit yield. However, its estimation is often based on subjective evaluations of fruit growers. This study explores the application of the YOLO framework for flowering intensity estimation. YOLO is a popular computer vision solution for object-detecting tasks. It was applied to detect flowers in different studies. Still, it requires manual annotation of photographs of flowering trees, which is a complex and time-consuming process. It is hard to distinguish individual flowers in photos due to their overlapping and indistinct outlines, false positive flowers in the background, and the density of flowers in panicles. Our experiment shows that the small dataset of images (320 × 320 px) is sufficient to achieve an accuracy of 0.995 and 0.994 mAP@50 for YOLOv9m and YOLOv11m using aggregated mosaic augmentation. The AI-based method was compared with the manual method (flowering intensity estimation, 0–9 scale). The comparison was completed using data analysis and the MobileNetV2 classifier as an evaluation model. The analysis shows that the AI-based method is more effective than the manual method.

1. Introduction

Climate change is increasingly affecting agriculture, but at the same time, the world population is growing, which means that it is necessary to promote the efficiency and productivity of farming. As a result, data-based decision-making plays a crucial role in agriculture, ensuring efficient farming. This also applies to the management of commercial fruit orchards. During the spring, auditing flowering intensity is important for understanding the necessary actions of orchard management [1]. Flowering influences orchard fertilizing and pruning strategy and the necessity of flower and fruitlet thinning that affects fruit quality and stability of yielding year by year, etc. [2,3]. In the end, such information gives the possibility to be better prepared for harvest time—arranging workforce, managing storage facilities, selling activities, etc. Therefore, it is necessary to have tools already in the springtime to estimate flowering intensity and to predict fruit yields [1].

Yield prediction can be performed using regression models based on the detected number of flowers, fruitlets, or fruits depending on prediction time. Therefore, yield prediction depends on the accuracy of object counting, which is completed by artificial intelligence (AI) using photos collected by agrobots and unmanned aerial vehicles (UAVs). However, the real number of objects must be predicted because AI counts only visible objects, but many flowers, fruitlets, and fruits are hidden in the foliage of trees (observational error) or are invisible from the side of image capture. The real number of objects can be estimated using a linear regression model processing the number of objects detected by computer vision. This approach was applied by Lin, J. et al. (2024) [4] to count lychee flowers and by Cai, E. et al. (2021) [5] to count panicles of Sorghum plants. The linear regression model was applied to predict the real number of fruits too, as follows: Nisar, H. et al. (2015) [6] predicted the yield of dragon fruits, but Brio, D. et al. (2023) [7] predicted the yield of apples and pears.

Brio, D. et al. (2023) [7] completed a comprehensive analysis of fruit counting using computer vision. Their experiment showed that the Pearson correlation for manually counted fruits per tree and the count detected by a deep learning model trained from image analysis was up to 0.88 and 0.73 for apples and pears, respectively. Meanwhile, Farjon, G. et al. (2019) [8] indicated a Pearson correlation of 0.78–0.93 for counting apple flowers. But, Linker, R. (2016) [9] investigated the accuracy dependence on two or six photos per tree captured from different sides. He concluded that the usage of images from only one side of the tree does not worsen the results significantly.

While the observation error depends on environmental conditions and remote sensing, the measurement error depends on the accuracy of AI. The modern development of AI primarily relies on data-based methods. For example, Brio, D. et al. (2023) [7] applied YOLOv5s6 and YOLOv7 for apple and pear counting, but Lin, J. et al. (2023) [4] applied YOLACT++ for litchi (Litchi chinensis) flower counting. However, the data-based solutions require large amounts of data to train AI. Meanwhile, the annotation of flowers is a time-consuming process because it is hard to distinguish individual flowers in photos due to the low resolution of images, noise-flowers in the background, and the density of flowers in panicles. Our pilot test showed that image annotation takes approximately 15 min per image (320 × 320 px) with flowers of apples.

The aim of this study is to develop a simple method to train YOLO models for flowering intensity estimation. The objectives of this study are as follows: (1) to prepare a small dataset of flowering trees; (2) to train YOLO models for flowering intensity evaluation; and (3) to compare the AI-based method with the manual method. The use-case of the experiment is flowering apple trees. Two popular object detection architectures were selected for the experiment, YOLOv9 [10] and YOLO11 [11].

In this study, we present the method of flowering intensity estimation using computer vision. The method is based on the annotation of small images (320 × 320 px), which are randomly grouped into 640 × 640 px images using Python script before YOLO training with an input layer of 640 × 640 because the input layer of pretrained YOLO models is 640 × 640. The experiment showed that 100 images (320 × 320 px) are enough to achieve accuracy of 0.995 and 0.975 mAP@50 and 0.974 and 0.977 mAP@50:95 for YOLOv9m and YOLO11m. When YOLO models were trained, the accuracies of the manual and AI-based methods were compared using data analysis and a MobileNetV2 classifier as an evaluation model.

2. Materials and Methods

2.1. Flowering Intensity Estimation Model

Productivity, meaning the fruit yield, depends on the flowering intensity of fruit trees. However, in some species, excessive flowering and the related fruit set may directly reduce fruit size (like apples) or even lead to total production loss. The biannual production can be induced by the result of the over-fruit set as well. For example, in the case of Citrus sinensis (L.) Osbeck, an increase in flowering beyond intermediate intensity results in a reduction in both the initial fruit set and final fruit yield at harvest. This inverse relationship between flowering and yield is not unique to citrus fruit. As a result, the economic disadvantages of excess crop load have resulted in considerable research on fruit thinning and widespread commercial application of this practice [2,3,12]. Therefore, the fruit growers must know the optimal flowering intensity for specific cultivars. Appropriately, object detection models (CNNs) are not the final product; they are only measurement instruments to precisely count the number of flowers to associate it with the optimal flowering intensity or to predict fruit yield—the conceptual model is depicted in Figure 1. Object detection models are called YOLO models in Figure 1 because they are the most popular CNN architecture nowadays. Considering the model in Figure 1, the AI must simply answer the question “Is flower thinning required?”. Meanwhile, the prediction of a fruit yield must be completed after flower thinning (see Figure 1).

2.2. Dataset Collection

The modern computer vision is based on the application of CNNs and deep learning. Therefore, the training dataset was required for the experiment. The photos were collected during blooming at the Institute of Horticulture (LatHort), coordinates WGS84 (Lat: 56.6092713, Lng: 23.3064627). The photos from several apple trials located side by side in equal soil and orchard management and climatic conditions were used to model development, including fifteen genetically and morphologically different cultivars mostly on dwarf rootstocks like B.396 [13,14]. The orchards of the trial were managed according to a sustainable (integrated) growing system. The orchard had no irrigation.

The collected photos were grouped by the flowering intensity of the trees. The flowering intensity was estimated in points 0–9, where 0 points mean no flowers observed; 1 point—flowers have up to 10% of possible points of their development; 2 points—flowers have 10% to 20% of possible points of its development, etc.; 8 points—flowers have 70% to 85% of possible points of its development; 9 points—flowers have 85% to 100% of possible points of its development. The examples of photos are presented in Figure 2.

2.3. Flowering Intensity Estimation Using Computer Vision

Flowering intensity can vary between cultivars (thickness of branching and crown, volume of crown), and the system of forming crowns can affect it. Therefore, the flower detection CNN can be trained once for each species (apples, pears, cherries, etc.), and then it can be applied to specific cultivars and commercial orchards (to the final environment) to precisely estimate the optimal flowering intensity considering the model depicted in Figure 1. The more advanced approach is transfer learning, where the general CNN is tuned to a specific cultivar or the stages and types of flowers.

Nowadays, the most popular object detection architecture is YOLO, which was applied in our experiment. However, it is a time-consuming process to annotate images with the categories “flowers” and “buds” because visual objects are small and hardly visible, as well as their number can be huge. Some simple low-cost method is required to save annotation time. In this study, we applied the following method: we annotated 100 images with the size 320 × 320 px (see Figure 3a), which were cut out from original images with the size 3008 × 2000 px without decreasing the resolution. YOLO frameworks traditionally provide pretrained models with an input of 640 × 640 px. Therefore, 320 × 320 px images can be simply grouped to create 640 × 640 px images using an aggregated mosaic (see Figure 3b). Considering the probability theory, 100 images can provide 94,109,400 permutations or 3,921,225 combinations. Each image can be independently pre-processed by augmentation methods before new image construction. The aggregated mosaic was used by Zhao et al. (2021) [15] to create a training dataset for object detection in remote sensing images. The objects in remote sensing images are typically sparsely distributed, and the aggregated mosaic prevents the blank training samples. The images of flowering trees have a similar problem with sparsely distributed panicles. The aggregated mosaic is like the CutMix augmentation strategy. Yun et al. (2019) [16] experimentally showed that CutMix overcomes MixUp and CutOut methods providing an accuracy increase of 2.3% for ImageNet. It is a common mosaic augmentation; however, we underlined it as a low-cost method because other authors try to annotate the whole images of flowering trees, which is a time-consuming process. The concept of the training method is depicted in Figure 4. In our case, 15 min were required to annotate one image (320 × 320 px) or 25 man-hours for the dataset with 100 images. The tool makesense.ai was applied for annotation tasks. The dataset is available in Kaggle under a CC-BY 4.0 license [17].

2.4. Machine Learning and Software

The experiments were completed using the video card MSI GeForce RTX4070 Ti (Micro-Star International, New Taipei City, Taiwan) with 7680 CUDA (NVDIA, Santa Clara, CA, USA) cores and 12 GB GDDR6X (Micron Technology, Boise, ID, USA). The data analysis and CNN training were completed using Jupyter Notebook 7.3.3, Python 3.10, and PyTorch 2.5. The pretrained YOLOv9m and YOLO11m models were selected for the experiment to calculate the number of flowers. The default training settings of the YOLOv9 and YOLO11 frameworks were applied. These models were selected based on our experience and previous experiments. However, other architectures and models can be used too. Meanwhile, MobileNetV2 pretrained on ImageNet was selected for flowering intensity classification. We did not tune the models because our goal was to evaluate the potential of the proposed methodology. In the future, the methodology must be applied to specific cultivars considering the model in Figure 1.

3. Results

3.1. Detection Accuracy of Flowers and Flower Buds

The middle-size models, YOLOv9m and YOLO11m, were applied in the experiments. Each model was trained five times. The mAP@50 and mAP@50:95 accuracies are presented in box-plot diagrams (see Figure 5 and Figure 6). The flowers were detected with an accuracy of 0.995 mAP@50 in all cases. Therefore, there are lines in Figure 5b. The YOLOv9m showed the best accuracy of mAP@50 0.995, but YOLO11m showed the best accuracy of mAP@50:95—0.977.

3.2. Flowering Intensity Estimation: Manual vs. Computer Vision

When the object detection models were trained, it was interesting to compare the accuracy of flower intensity estimation between the manual (subjective and making notes on paper, etc.) method and computer vision. We had the categories of flower intensity defined by a 10-point system (0–9 points). We did not have the information about the precise number of flowers for each tree. Therefore, we applied the trained YOLOv9m and YOLO11m models to calculate the number of flowers and flower buds for each category defined subjectively by fruit growers. The obtained numbers of objects for each category are depicted in Figure 7. The average number of objects was calculated for each category of flowering intensity (average of averages). The filter of two standard deviations was applied to exclude anomalies. It was required because each YOLO model provided a different number of objects, and this approach provides the possibility to extract the opinion of the larger group. The dataset with flowering trees is available in Kaggle under the CC-BY 4.0 license [18]. A background was masked to exclude the impact of flowers from other trees visible in the image. In the beginning, we applied all images without dividing them into cultivars.

The categories I0–I2 and I7–I9 are allocated considering their sequence. However, other middle categories are mixed, e.g., I5 is smaller than I4, but I3 is greater than I4 and I5. As well as this, the categories {I0–I2, I3–I5, I6–I7, and I8–I9} are tightly allocated and neither are distributed with equal intervals. It depicts the perception error of fruit growers. Therefore, we completed additional analysis measuring the percentage of the visual overlay by flowers and flower buds over the apple trees to imitate the visual perception of fruit growers (see Figure 8). Using the previous approach, we calculated the average percentage of the visual overlay by flowers and flower buds (see Figure 9). The data analysis showed similar results (see Figure 9)—the categories are mixed like in Figure 7. This depicts the perception error of fruit growers.

Considering that flowering intensity can vary between cultivars, we repeated the analysis by grouping the data by cultivars. We selected the largest groups of cultivars, which were presented in the dataset. This was because the photos were collected within one season and the cultivars did not contain the representatives of all the categories of flowering intensity. The data are presented in comparative form (see Figure 10), where the left column presents the categories by the number of flowers, but the right column presents the categories by the percentage of the visual overlay. Figure 10 depicts similar problems as in Figure 7 and Figure 9—the flowering intensity categories are mixed. However, the right column (visual overlay) contains more correctly allocated categories considering their sequence than the left column (the number of flowers). It shows that fruit growers classify flowering intensity based on the visual perception of flowering trees.

Now, we must imitate the work of artificial intelligence. The clustering method “Jenks Natural Breaks” (JNB) is applied to estimate the categories of flowering intensity. JNB searches the optimal ranges of values for each category considering the principle that the standard deviation within one category must be minimal, but the distance between two categories must be maximal. This clustering method is popular among users of geographical information systems (GISs) because it is applied to create thematical color maps. If fruit growers try to estimate flowering intensity, it is a percentage from the maximal possible based on their experience and subjective opinion. JNB is based on statistics and is closely related to Likert scales. It does not calculate percentage values; it identifies statistical clusters. Therefore, JNB is suitable for regrouping subjective categories into data-based categories. In the experiment, JNB was applied to the values of the visual overlay with a YOLO confidence of 40%. The result of artificial classification is depicted in Figure 11a. Now, the flowering intensities are allocated with intervals. However, the related number of objects is better; it is not so ideal (see Figure 11b). There is a large gap between I9 and groups I7–I8, but other categories contain better intervals although they are relatively close. Thus, the data analysis of categories shows that the AI solution is better than the manual method (see Figure 7 and Figure 11b). Maybe it is comfortable and natural for people to estimate flowering intensity by perception of visual flower clusters; the data-based approach is simpler for AI because object detection models can directly calculate the number of objects. Therefore, it is more correct to apply JNB clustering to the number of flowers and flower buds directly (see Figure 11c).

Additionally, an interesting fact is visible in Figure 7 and Figure 11b,c: the categories try to make four groups. We can imitate the visual perception with four categories using MobileNetV2 architecture, which is suitable for image classification. The classification of four categories provides more accurate results than the solution with ten categories of subjective evaluation (see

Table 1. Classification of flowering intensity using MobileNetV2.

	4 Categories
Image Size	Manual	JNB
256 × 256	0.282	0.444
512 × 512	0.615	0.722
1024 × 1024	0.641	0.805
	10 categories
Image size	Manual	JNB
256 × 256	0.103	0.300
512 × 512	0.205	0.350
1024 × 1024	0.385	0.450

). Using the MobileNetV2 classifier as a comparison instrument, we can conclude that the 10-point system is inaccurate due to its complex visual perception (influence of subjectivity—meaning only subjective values without a constant control tool, depending on the lack of consistency due to tiredness degree).

4. Discussion

The number of fruits is correlated with the number of flowers produced by each tree. Therefore, this parameter is important for fruit growers to plan and allocate human and economic resources during the harvesting season [19]. Meanwhile, the flowering intensity is an essential parameter for flower thinning, fruit yield, and fruit quality [3]. Traditional flower counting is completed manually by fruit growers based on their experience and requires frequent observation and professional knowledge. However, this approach is time-consuming and prone to errors [4]. Therefore, different authors studied the possibility of automating flower counting using computer vision [3,5,19,20]. However, it is not a trivial object detection problem, and plenty of challenges are mentioned by researchers.

Firstly, there are the tight clusters of flowers. It is extremely difficult for the model to differentiate objects. This issue is not only a problem for the model, but the annotation by hand of the dataset is also very confusing and difficult [20]. Secondly, the presence of background trees and flowers introduces further complexity, as it can potentially confuse the recognition models [19]. Additionally, the researchers mention the accuracy limitations of the YOLO architecture to detect small and high-density objects such as flowers [3,19]. Lin, J. et al. (2024) [4] proposed a special framework with YOLACT++, FlowerNet, and regression models to improve flower counting through Density Map. Meanwhile, Estrada, J.S. et al. (2024) [19] compared the accuracy of YOLOv5, YOLOv7, and YOLOv8 with Density Map, where Density Map showed better results.

Of course, it is important to improve object detection architectures to search small and high-density objects such as flowers. However, there are doubts about AI accuracy. The image depicts only one side of the tree, which can be analyzed by an object detection model. Other flowers must be predicted by a regression model or precise 3D reconstruction of the tree is required. A similar problem was studied by Lin, J. et al. (2024) [4] and Nisar, H. et al. (2015) [6], who applied regression models to predict the real number of objects. Meanwhile, Linker, R. (2016) [9] concluded that the usage of images from only one side of the tree does not worsen the results significantly. Regression models are constructed based on statistical tuples (x, y), where x is the input value, and y is the predictable value. Therefore, yield predictors must be tuned for each orchard individually to consider the local climate and orchard management. Another important fact is the accuracy of manually counted flowers. For example, Lin, J. et al. (2024) [4] completed the correlation test between the image flower count and the actual flower count investigating that the actual count of single-panicle flowers is highly positively correlated with both the manually annotated image count (p < 0.01, r = 0.941) and the predicted flower count from the FlowerNet model (p < 0.01, r = 0.907). Meanwhile, the annotation of flowers is a subjective process, and a data annotator is restricted to the picture visible on the image. Additionally, if the photo shot is completed from another side, the resulting number of flowers will be different. Considering the linear regression model, the prediction error will be proportional to the input error. For example, Lin, J. et al. (2024) [4] mentioned that the results obtained from Density Map-based methods can exhibit significant variation when images are captured at different distances. Also, YOLO models work using the parameter “confidence”: the higher the confidence is set, the smaller the number of objects returned by the model. A visual example is presented in Figure 7 and Figure 11b. In laboratory conditions, it is simple to find the optimal confidence to obtain minimal errors for a regression model. However, it is not possible in field conditions, where confidence must be strictly defined for a final product.

Another ambiguity is flowering intensity itself. The flowering intensity must be estimated for each cultivar individually, which is performed based on the experience of fruit growers [4], [12]. For example, Chen, Z. et al. (2022) [3] trained the modified YOLOv5 model to detect six stages of apple flower: (1) bud bursts; (2) tight clusters (green bud stage); (3) balloon blossom (most flowers with petals forming a hollow ball—stage 59 by BBCH scale of pome fruits); (4) king bloom (first flower open—BBCH 60); (5) full bloom, where at least 50% of flowers open and first petals are falling (BBCH 65); and (6) petal fall—end of flowering (BBCH 69). It was conducted with the objective of identifying the blooming peak day because the timing of flower thinning is generally decided by the ratio of “king bloom” clusters and “full bloom” clusters. Meanwhile, Lin, J. et al. (2024) [4] did not differentiate the stages of flowers; the collected dataset contained two litchi flower varieties, Guiwei and Feizi Xiao, and images of the first-period male litchi flowers during the early flowering stage. Estrada, J.S. et al. (2024) [19] did not identify different stages of flowers too. Lee, J. et al. (2022) [20] studied one cultivar of apple trees (Ambrosia) and identified two types of flowers: “Fruit Flower Single” and “Fruit Flower Cluster”. In our study, the fruit growers estimated flower intensity by a 10-point system (0–9 points). So, all developers of flower detectors [3,4,19,20] wanted to improve manual flower counting (subjective visual estimation) by replacing it with a precise tool. However, all developed models present different flower intensity estimation techniques, which are influenced by local fruit-growing traditions and their understanding by AI developers.

The same initiative motivated our study—to automate flower intensity estimation to assist fruit growers with a smart tool. However, the annotation of flowers and buds is a time-consuming process. Our pilot test showed that image annotation takes approximately 15 min per image (320 × 320 px) with flowers of apples. Considering Adhikari et al. (2018) [21], the manual annotation takes about 15.35 s per bounding box and its correction for the indoor scene. In our dataset, the average number of objects was 26.42 and the maximum was 92, which would be 6 min 25 sec and 23 min 32 sec, respectively, but the images of flowers are more complex. Therefore, the double long time (15 min) is comparable with the results presented by Adhikari et al. (2018) [21]. Due to a time-consuming process of the annotation, we investigated the simple training method (see Figure 4) and achieved excellent accuracy for YOLOv9m, 0.993–0.995 mAP@50 and 0.967–0.974 mAP@50:95, and for YOLO11m, 0.992–0.994 mAP@50 and 0.970–0.977 mAP@50:95.

However, if the ideal flower counter is developed, knowledge about the number of flowers is not sufficient for efficient decision-making because there are plenty of factors that can impact fruit yield, like an orchard system itself. Orchards in uncovered areas are directly affected by external environmental factors—excessive heat, excessive drought, excessive cold, storms, etc.—which have been increasingly observed in recent years. Therefore, risk-based management can be applied in this situation. This means that a risk management plan is developed based on which the yield forecast is created throughout the season. The risk management plan identifies possible risks, classifies risks, and assesses the impact and likelihood of risks. Most importantly, for this information to be used in yield forecasting, based on the results of the risk assessment and the actions determined for each risk (mitigation, buffering, acceptance), the impact of each risk on the yield is quantified. For example, if the storm speed is 35 m/s, the impact on the yield reduction is 15%. In this case, it is assumed that if this risk occurs, the yield will be 15% lower than was initially determined by counting flowers. As mentioned earlier, yield forecasting is essential for making accurate decisions to plan sales, the number of employees needed, etc. Considering the risks, it is not possible to obtain precise yield prediction without monitoring in real time. Meanwhile, the high precision requires the 3D reconstruction of flowering trees, which is challenging and costly for modern industry. Zhang et al. (2023) [22] presented a solution based on the application of UAVs. The flower intensity estimation is based on the image captured from the top (sky), tree segmentation, and the calculation of flower pixels. This solution provides the next advantages: (1) it automatizes yield monitoring by using UAVs that fly over trees; (2) the images do not have noise-flowers in the background; and (3) considering Linker, R. (2016) [9], the usage of images from only one side of the tree does not worsen the prediction results, and the linear regression models can be constructed based on pixel indices. However, this approach has disadvantages too—the monitoring distance interrupts to capture the flower buds and provide flower stage recognition. Considering the method, Zhang et al. (2023) [22] tested the accuracy based on man-made flowering intensity estimation (1–9 points). As it was shown, the manual evaluation is subjective and provides the perception error. Therefore, two technologies must be developed together. Replacing the manual flowering intensity estimation with computer vision, it will be possible to develop more precise linear regression models for UAV solutions. Also, YOLO models can be embedded into agrobots for automatic flower thinning, which is another function and service for horticulture.

5. Conclusions

Flowering intensity is an important parameter to predict and control fruit yield. We completed a study related to flower intensity estimation using computer vision. The flower intensity estimation using object detection models is not a trivial problem because it is extremely difficult for the models and image annotators to differentiate objects due to the tight clusters of flowers. However, our experiment showed that YOLOv9m and YOLO11m models can be trained with accuracy of 0.995 and 0.994 mAP@50 using 100 320 × 320 px images. This method is based on the aggregated mosaic augmentation, which combines the 320 × 320 images into 640 × 640 images. This approach significantly saves time for image annotation, and it is more comfortable to annotate small images rather than the whole tree. Additionally, we completed data analysis and proved that computer vision must be developed for precise estimation of flowering intensity because the man-managed visual evaluation method is not accurate due to the perception error of fruit growers. This study was conducted using a dataset with fifteen cultivars. The study dataset did not contain the representatives of cultivars with all categories of flowering intensity because the experiments were completed with one-year photos. Therefore, the precision of the results can be improved by repeating this study using the dataset where one cultivar will have all categories of flowering intensities. Additionally, there are different models of flowering intensity estimation and flower thinning techniques in the world. It is required to investigate the most appropriate method for artificial intelligence. The precision of the manual estimation was evaluated using the trained YOLO models as a measurement instrument. More precise results could be obtained based on the manual annotation of all the images. The experiments were completed in laboratory conditions using the images collected in natural conditions. At this moment, the practical application of YOLO models by replacing the manual methods is doubtful due to the ambiguity of the confidence level. However, the development of computer vision for flower intensity estimation is strongly important because the manual method is subjective and a data-based approach is required to make precise decisions.