Next Article in Journal
Gesture Recognition Based on Multiscale Singular Value Entropy and Deep Belief Network
Next Article in Special Issue
A Cloud-Based IoT Platform for Precision Control of Soilless Greenhouse Cultivation
Previous Article in Journal
Autonomous Power Decision for the Grant Free Access MUSA Scheme in the mMTC Scenario
Previous Article in Special Issue
Assessing Effects of Salinity on the Performance of a Low-Cost Wireless Soil Water Sensor
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:

UAV-Based RGB Imagery for Hokkaido Pumpkin (Cucurbita max.) Detection and Yield Estimation

Remote Sensing Group, Institute of Computer Science, Osnabrück University, 49074 Osnabrück, Germany
Agronomy and Crop Science, Kiel University, 24118 Kiel, Germany
Faculty of Agricultural Sciences and Landscape Architecture, Osnabrück University of Applied Sciences, 49076 Osnabrück, Germany
Author to whom correspondence should be addressed.
Sensors 2021, 21(1), 118;
Submission received: 4 December 2020 / Revised: 23 December 2020 / Accepted: 23 December 2020 / Published: 27 December 2020
(This article belongs to the Special Issue Sensors in Agriculture 2020)


Pumpkins are economically and nutritionally valuable vegetables with increasing popularity and acreage across Europe. Successful commercialization, however, require detailed pre-harvest information about number and weight of the fruits. To get a non-destructive and cost-effective yield estimation, we developed an image processing methodology for high-resolution RGB data from Unmanned aerial vehicle (UAV) and applied this on a Hokkaido pumpkin farmer’s field in North-western Germany. The methodology was implemented in the programming language Python and comprised several steps, including image pre-processing, pixel-based image classification, classification post-processing for single fruit detection, and fruit size and weight quantification. To derive the weight from two-dimensional imagery, we calculated elliptical spheroids from lengths of diameters and heights. The performance of this processes was evaluated by comparison with manually harvested ground-truth samples and cross-checked for misclassification from randomly selected test objects. Errors in classification and fruit geometry could be successfully reduced based on the described processing steps. Additionally, different lighting conditions, as well as shadows, in the image data could be compensated by the proposed methodology. The results revealed a satisfactory detection of 95% (error rate of 5%) from the field sample, as well as a reliable volume and weight estimation with Pearson’s correlation coefficients of 0.83 and 0.84, respectively, from the described ellipsoid approach. The yield was estimated with 1.51 kg m−2 corresponding to an average individual fruit weight of 1100 g and an average number of 1.37 pumpkins per m2. Moreover, spatial distribution of aggregated fruit densities and weights were calculated to assess in-field optimization potential for agronomic management as demonstrated between a shaded edge compared to the rest of the field. The proposed approach provides the Hokkaido producer useful information for more targeted pre-harvest marketing strategies, since most food retailers request homogeneous lots within prescribed size or weight classes.

1. Introduction

Pumpkins (Cucurbita spp.) are produced on 2 million ha of land worldwide and becoming increasingly popular across Europe [1]. In Germany, the harvested area steadily increased over the last two decades and reached more than 5000 ha in 2018 (Figure 1). Whereas the global yield was on a constant level of about 13.3 t ha−1, in Germany, yields are on a slight decrease was observed [1]. This could be due to a shift towards more consumer-oriented production of smaller fruits (<1.5 kg).
The origin of pumpkin domestication was in the Americas, where several Cucurbita species were cultivated in the pre-Columbian era; together with maize, they were later brought to Europe [2]. Breeding progress for earliness and adaptation to temperate climate helped to spread the cultivation across higher latitudes. Curcubita has manifold forms and properties, and the very popular “Hokkaido” pumpkins belong to the Curcubita maxima species within the “Hubbard” varietal group [2]. Whereas, in Europe, only a few species are produced for professional food retailing, most of the manifold pumpkins (or winter squash) are cultivated by private gardeners and in developing countries by local farmers from landraces and saved seeds.
There are two main directions of cultivation: for human consumption of the whole plant or for high-quality oil production (from the seeds), as well as a smaller share as medical plants due to their secondary ingredients (e.g., high amount of carotenoids) [2,3]. The seeds are a high-energy source and known as snack in some regions of the world [4]. The production for European food retail is focusing on small individual fruits since about 0.5 to 1.5 kg. Large pumpkins (>2.2 kg) can successfully be commercialized around Halloween for carving.
Due to a more diverse fruit development in pumpkin fields compared to crops planted in higher stand densities, precise yield predictions are difficult but of great interest for logistic planning and marketing activities. Non-destructive pre-harvest yield estimations with detailed information about size, weight and number of fruits can help farmers to improve their commercialization strategies. Unmanned aerial vehicle (UAV)-based remote sensing approaches are a cost-effective way to gain such information.
In recent years, high-resolution UAV data have become an import source for many agricultural applications, such as plant detection [5,6], plant monitoring [7,8], or yield prediction [9,10]. Especially, as drones and basic RGB camera systems have been getting more affordable, the use of drone systems has increased in the agricultural sector.
Current studies have demonstrated the suitability of UAV data to detect and count fruits for yield estimation, including strawberries [11], apples and oranges [12], citrus fruits [13], or melons [14]. However, from our state of knowledge, almost no studies are available on the detection of pumpkin fruits for yield estimations from UAV data. The main challenges in the above-mentioned papers were related to the detection of single fruits which overlap [11] or were partially hidden under leaves [13]. To overcome those restrictions, the studies used techniques from the currently very successful field of deep learning, allowing the shape of a fruit to be learned using a neural network. However, these models usually require a large amount of training data to outperform classical methods [15] and are very computationally expensive to train.
The aim of this study was to develop a pipeline to detect and count Hokkaido fruits for yield prediction using high resolution UAV RGB imagery. Therefore, we used classical methods from the field of machine learning and computer vision, which have been successfully applied in object detection tasks [10,16]. In order to address the issue of overlapping and close-by fruits, we implemented a simple conditional and thresholding strategy to split those fruits. For shape reconstruction, we used geometric features. Besides this, the estimated number and yield distribution of the fruits within the study area was investigated and evaluated to derive agronomic implications.

2. Materials and Methods

2.1. Field Management

The experimental site was located on a 0.25 ha field in North-western Germany (52.33° N, 7.97° E WGS84, 65 m asl) close to Osnabrück on a Stagnic Cambisol soil type, following the World Reference Base (WRB) classification [17] with low to medium yield potential. The climate is characterized as temperate oceanic (Cfb according to Köppen-Geiger classification) [18] with a mean annual air temperature of 9.4 °C and 883 mm annual precipitation. The experimental growing season received average precipitation but with a considerably uneven distribution, and temperatures were 1.6 °C above the long-term average (Figure 2).
The previous crop on the field was maize, and all agronomic operations are listed in Table 1. The Hokkaido pumpkins (Cucurbita max. Duchesne cv. Uchiki Kuri) were planted as seedlings in growth stage 12 according to BBCH [20] with a semi-automated planting machine in mid-May. Harvest took place at maturity after 130 days.

2.2. UAV Image Data

High-resolution UAV image data were acquired on 23 June 2017 under cloudless weather conditions to ensure similar illumination conditions. For the aerial survey, a DJI Phantom 4 was used, which features an on-board RGB camera with a sensor resolution of 12 megapixels and a focal length of 5.74 mm. A flight altitude of 46 m above ground level was chosen. This allowed a spatial resolution of 1 cm and an overall flight time of about 7 min to cover the study area.
For georeferencing, ten field targets were placed evenly in the study area, which were used as ground control points (GCP) in the post-processing step. The target centers were located using a differential GPS (bi-frequency GNSS receiver) based on the German SAPOS correction service for precise positioning in Real-Time Kinematic (RTK) mode.
The flight was conducted at noon so that shadowing was reduced to a minimum. During the flight, we used GPS and GLONASS positioning (p-mode) for platform stability. The camera was oriented vertically using the integrated three-axis gimbal. A total of 106 images were acquired with a side and front overlap of 75%, respectively.

2.3. Field Sampling

After the UAV flight, a total of 100 randomly selected Hokkaido pumpkins were measured for diameter and height, and the exact position of each pumpkin fruit was located by differential GPS. The diameter refers to the largest horizontal extension of the pumpkin, whereas the height is determined by the distance between the stem and the blossom end. During harvest, which was carried out a few days after the flight, another 40 pumpkins were measured following the same collection procedure. For yield estimation, we also determined the weights of these pumpkin fruits.

2.4. Methodology for Fruit Identification

The methodology of this work consisted of numerous steps, including image pre-processing (1), pixel-based image classification (2), classification post-processing for single fruit detection (3–4), and fruit quantification (5). All steps of the methodology were implemented in the programming language Python (version 3.7). An overview of the proposed pipeline is illustrated in Figure 3.
In the first step, we created an orthophoto mosaic from the collected UAV images based on photogrammetric processing using the Agisoft Metashape software (Version 1.5.3). In addition, the image was georeferenced using the located in-field targets. Then, we processed the collected UAV images with the Agisoft Metashape software (Version 1.5.3), including image stitching and georeferentiation using in-field targets. For fruit detection and counting, a classification approach was initially used. Therefore, representative pixels of the classes fruit and non-fruit were manually digitized from the generated orthomosaic. The samples were then divided equally into a training and test set. For each class, 1000 pixels were randomly selected from the test data to evaluate the classification algorithm.
The pixel-based classification of the Hokkaido fruits was based on a binary random forest (RF) algorithm, which consists of large number of decision trees, whose individual results are combined into a single classification result. Since multiple decision trees are used, this approach is considered to be robust against overfitting, as well as unbalanced class sizes [21]. The RF enhances common decision trees using the methods of bagging and random feature selection. The first implies that each decision tree is assigned only to a random selection of training data, whereas the random feature selection reduces the number of input variables randomly [22]. The classification relies on the assignment probabilities of all trees, which are determined by a majority vote. This means that a certain input is assigned to that class, which is predicted by most of the trees.
In this study, the RF classifier was instantiated with a total of 500 trees, since studies have shown that the error term remains nearly constant with this number of trees [23]. To separate the classes within a decision tree, we used the Gini Index, which measures the average gain of class purity by splits of a given variable [24]. The accuracy assessment of the classification model was based on the independent test data. The classification quality was determined using a confusion matrix, including overall accuracy (OA), producer’s accuracy (PA), and user’s accuracy (UA).
In order to overcome misclassifications in the binary classification map, we applied a sequence of morphological filters. The idea behind this approach was to remove isolated pixel with high uncertainty, as well as to increase the homogeneity of the segmented fruit objects in shape and size. Morphological filters are Boolean filters, which are very effective for smoothing binary images [25]. In this context, two basic morphological operators are often used: erosion and dilation. The dilation filter expands the size of an object by adding pixels at the object boundary, which also allows gaps within objects to be closed. In comparison, the erosion filter reduces the size of the foreground. Therefore, objects of small size are removed, and object boundaries are sharpened [26]. Both operations produce a homogeneous appearance of objects, depending on the shape of the object and the given filter size. However, the application of this operators also leads to a change in object size. To keep the approximated object sizes, the operators opening and closing are often applied, which uses the operators erosion and dilation in sequence. Opening is defined as the application of erosion, followed by a dilation. Closing is determined as dilation, followed by erosion. Opening and closing have the same smoothing properties as erosion and dilation, respectively [25]. In this work, we initially applied the opening operation to eliminate classification noise, followed by closing for smoothing the remaining objects. For the morphological filtering, a kernel size of 3 × 3 pixel was specified.
Even with the application of morphological filtering, the derivation of the correct shape and size of the segmented Hokkaido fruits can be limited if they are partially covered by leaves or stems. To reconstruct the actual shape of the segmented fruits, the geometric form of an ellipse was applied to every segmented object.
The ellipse was chosen because the Hokkaido fruit is normally round-shaped, and its height is usually smaller compared to its diameter. This results in an oval shape, which can be approximated by an ellipse. To apply the ellipses to the segmented objects, every object had to be identified in the image first. This was accomplished by using the method findContours by Suzuki et al. [27] of the open-source computer vision library (OpenCV). In the next step, for every fruit object, an ellipse was approximated using the fitEllipse algorithm. This function uses the principle of least squares to fit an ellipse to a selection of points on a given two-dimensional plane [28].
Based on the segmentation steps presented, nearby fruits could be classified as joint objects. For the quantification of the Hokkaido fruits, it was essential to split an object accordingly to the number of fruits it represents. In order to select those objects, two conditions were tested: Firstly, the aspect ratios of the applied ellipses were calculated. It was assumed that several fruits within an ellipse result in a stretched geometry, leading to an increased aspect ratio, as illustrated in Figure 4 (left). Secondly, the difference between the fruit ellipses and their corresponding object segmentation after the morphological filtering was calculated. Here, the approximation errors of the ellipses compared to the more precise object segmentations should be highlighted to reveal spaces between the nearby fruits, as shown in Figure 4 (middle). For both conditions, threshold values were applied to identify objects with multiple fruits.
To separate the identified objects, a bounding box was placed around these segmented objects before the ellipse has been applied (Figure 4, right). Then, the object was divided into three equally sized areas, which were always oriented parallel to the shorter length of the bounding box. Within the central area, a separation of the considered object was performed where the object had the lowest extension. After splitting, for every new generated object, an ellipse was approximated, as presented before. For separation of objects that represented more than two fruits, the above steps were repeated iteratively as long as the tested conditions matched a considered object. After this step, the total number of pumpkins in the field could be counted.
Besides the counting of fruits, this study aimed to derive the weights of Hokkaido fruits for yield prediction. However, the fruit weights could not be determined directly from the UAV data, due to their two-dimensional representation in the orthomosaic. Nevertheless, we assumed that the fruit weight is strongly related to the fruit volume, which can be approximated by a 3D oblate spheroid shape. An oblate spheroid is defined by a minor axis (c) and two major axes (a), resulting in symmetrical form with flattened height. This form is very similar to the shape of a Hokkaido, which is mostly round-shaped and squashed from above (Figure 5).
For volume approximation, the diameters and heights were determined of each fitted ellipse in the image and correlated with the major and minor axes of the spheroid using the semi-height and semi-diameter (Equation (1)).
V s p h e r o i d = 4 π 3   * a 2 * c   V p u m p k i n = 4 π 3   * ( d i a m e t e r 2 ) 2   * h e i g h t 2   .
For this, we defined the longer axis of the ellipse to be the diameter of a fruit, whereas the shorter axis was set as height. As the Hokkaido pumpkins predominantly lie sideways, we assumed that the height and the diameter could be plausibly derived from the image data. For the weight estimation, we calculated an empirical linear regression using the weights of the 40 harvested pumpkins and used the calculated regression equation to estimate the weight from the calculated volume of each generated ellipse. The performance of the regression model was validated using the coefficient of determination (r2).
The evaluation of the algorithm for fruit detection, as well as the estimation of fruit volumes and weights from UAV imagery, was performed through the comparison between the measured pumpkin fruits in the field and the corresponding fruits found in the image. For volume and weight estimation, the performance was investigated using the Pearson correlation coefficient. In order to evaluate incorrectly identified fruits in the UAV image, we used a random number generator (RNG) to select 200 objects from the orthoimage, which were then manually checked by human for accuracy.

3. Results

3.1. Descriptive Statistics of Field Data

Heights of the pumpkin fruits were measured in the range 6.5–19.0 cm (mean: 12.4 cm). Measurements of the pumpkin fruits diameters provided very similar values with slightly higher maximum (21.0 cm) and a mean of 13.8 cm, but pumpkin fruits with very low diameter of 6.0–7.0 cm were slightly overrepresented in the collected data set. The difference in mean of height and diameter is typical due to the slightly flattened shape of Hokkaido pumpkins. Weight varied over a wide range (142–3011 g; mean: 1.187 g), which was indicated by a standard deviation (SD) of 776.2 g. In case of pumpkin fruits weight, we sampled a high number of very light fruits with eight out of 40 pumpkins weighted less than 420 g. Detailed descriptive statistics of the investigated fruits are summarized in Table 2.

3.2. Accuracy Assessment of Random Forest Classification

With an overall accuracy of 94.7% most of the pixel were correctly identified. However, differences in accuracy were found between the two classes tested. With a PA of 97.3%, non-fruit pixels could be classified particularly successfully. Higher estimation errors occurred for the fruit class, where an PA of 92.1% was achieved. For the user’s accuracy, the results of the considered classes were reverse. In this way, a UA of 92.5% for the non-fruit class could be achieved, whereas the UA for the pumpkin class reached 97.3%.

3.3. Mapping of the Estimated Hookaido Pumpkins

Figure 6 shows the generated orthophoto, as well as a subset, of the classification map before and after the post-processing steps. In the UAV imagery, it can be seen that the investigated field was divided into an illuminated and a shaded region. Because of this, the fruit colors appeared very different in the orthophoto. In the illuminated part of the image, the fruits showed a very light reddish coloring. Due to the direct sunlight, the fruits were partially overexposed in many cases, resulting in a bright spot. Additionally, shadows appeared more frequently in this area of the image. In the shaded area, the fruits were evenly colored in an intense red, so that they could be clearly differentiated from the soil and the plants. In many cases, the fruits were only partially visible, as they were covered by leaves or stems.
Despite different representations of the Hokkaido fruits in the image, the majority of the fruits was correctly identified based on the initial RF classification. Especially, in the shaded area of the image, fruits were classified very precisely in size and shape. Fruits in the illuminated region were recognized with a similar accuracy, although some fruits, which were partially overexposed, revealed gaps. Besides this, bright plant stems were often incorrectly recognized as Hokkaido fruits. In comparison, these classification errors did not appear in the shaded area. Frequently, nearby pumpkins were merged into single segmented objects.
The classification map after the post-processing steps provided a significant improvement in the classification performance. Therefore, misclassifications could be removed, and gaps within the pumpkin objects, as well as errors of the pumpkin shape, have been decreased substantially. Furthermore, joined objects, which represent multiple pumpkin fruits, could be separated successfully in most of the cases. In only a very few examples, single segmented fruits were separated.

3.4. Quantification of Hokkaido Fruits

The presented mapping result revealed a high detection accuracy, which can be statistically confirmed, as the algorithm was able to detect 95% of the located and measured fruits from our field sampling. Additionally, it was shown that, from a sample of 200 detected pumpkins, all of them actually corresponded to real pumpkin fruits.
The regression model to predict the fruits weights from volumes indicated the strong relationship between calculated volume (from Equation (1)) and observed weight. The variance of the measured weights could be explained by more than 99% by the calculated pumpkin fruit volume. Consequently, the pumpkin fruit volume can be assumed as a very good proxy for the fruit weight.
Figure 7 (left) indicates a reliable relationship between the fruit volume observed and estimated from classified fruits in the image data with a correlation coefficient of 0.736 (significant at 99% level). If looking at the scatterplot in details, obviously, two samples do not follow the general trend. Fruit “A” was observed with a volume of 4040 cm³ but estimated with the much lower volume of 1885 cm3. The reason for this mismatch was that this fruit was partly covered by leaves (Figure 8, left). The other conspicuous case was fruit “B” with an observed volume of 1995 cm3 but estimated with a volume more than three times of the observed one (6839 cm2). In this case, insufficient image mosaicking was responsible for this “huge” fruit explaining the extremely high volume (Figure 8, right). If these two obviously erroneous cases were excluded from analysis, the correlation increases substantially to 0.826 (significant at 99% level).
Correlation between observed and estimated weight of fruits was found to be highly significant at 99% level with an r of 0.844. Although a certain scattering of data was observed, no noticeable outlier was identified (Figure 7, right).
The predicted fruit weights in the image were divided in weight classes, according to Reference [29]. Table 3 summarizes the total number of saleable (72.5%) and the share of non-marketable fruits from the mapping and classification procedures. Small pumpkins below 500 g can be utilized as animal feed.
Figure 9 shows the spatial distribution of pumpkins by weight classes according to the market practices with a gradient from more and heavier fruits to less and lighter pumpkins towards the south-eastern part of the field.
In Figure 10, an aggregated in-field distribution of number and weight per 3 × 3 m is presented. The number of fruits in the cells varied between one fruit and 25 fruits (Figure 10 left). The mean number of fruits for the entire field was determined with 1.37 fruits (SD: 1.19) per m2. A similar pattern of weight classes was derived from Figure 10 (right) with pumpkin weights ranging from less than one kg to more than 35 kg in the maximum. However, only one cell provided a weight higher than 26 kg. In relation to the field size, a mean weight of 1.51 kg m−2 (SD: 1.74) was estimated. Going into detail, one cell provided both the highest number of pumpkin fruits (25 per 9 m2), as well as highest yield (35.7 kg per 9 m2).
Looking at the south-eastern part of the field exhibits a different pattern compared to the rest of the field. In this area, the number of fruits, as well as the yield, were substantially below the rest of the field. This disparity shows the impact of the shading at the south-eastern edge with lower area-scaled average values. Besides this, no spatial pattern for the investigated field could be observed, neither for the number of fruits nor the yield.
Linking the number of fruits per cell to the predicted weight provided a substantial relationship with an r of 0.882 (significant at 99% level). Obviously, the number of fruits within a 3 × 3 m cell is highly positively correlated with the yield per cell.

4. Discussion

Many investigations described approaches for yield estimation from UAV-based images for crops, like cereals or oilseed rape, that form yield from stands with uniform single plants [30,31,32]. Those concepts often use total aboveground biomass and derive grain yield by known or estimated harvest index. In addition, applications for orchards were described for estimating fruit yield from permacultures, like apple or citrus, where the trees remain on the field [12,13]. For vegetables, stand height or biomass-based applications were described for eggplants, tomato, and cabbage under tropical conditions in India [33]. Kalantar [14] presented a successful individual fruit detection for yield estimation in a melon field where leave senescence started before harvest. Here, we presented a novel method for precise yield prediction based on single fruit identification, including size and weight classification, that allows more detailed harvest determination from a vegetable field with Hokkaido pumpkins between living leaves and stems.

4.1. Methodologic Performance

The pixel-based classification results using the RF classifier revealed high accuracies for both classes. However, it was found that fruit pixels were more often mistakenly classified as non-fruit than vice versa. One reason can be the high heterogeneity of this class, which includes different surfaces in various lightning conditions. In comparison, a very high user’s accuracy was achieved for the fruit class. This indicates that pixels of this class have a very high probability of belonging to this class [34], which is important for fruit counting as it reduces the number of wrong identified pumpkins as shown.
The mapping of the estimated pumpkin fruits revealed that the detection of single pumpkin fruits, as well as the approximation of their shapes, was achieved with high accuracies. RF classifications are described to achieve good accuracies for fruit detection [35], but the challenge increases in images from natural environments, as in our case. Regarding the RF classification approach, misclassifications in the illuminated field region between fruits with high reflectance (bright spot) and plant stems was limited, due to the similar color representation in the image, whereas other studies aimed on an additional shadow-removal step during the post-processing to reduce classification errors [36]. In our example, in the shaded area, noticeably fewer pixels were incorrectly assigned, which can be explained by the distinct color difference between the reddish fruits and the dark appearance of the soil and plants. The morphological filtering led to improved classification results. So, misclassified pixel of plant stems could be eliminated, and gaps within the segmented objects could be closed. This was possible as wrong classified pixel often occurred isolated or in a very narrow extend, which allowed them to be differentiated from the usually larger fruits.
The separation of neighboring pumpkin fruits was successfully applied in most cases. This issue usually poses a great challenge in single fruit detection [12,37]. A few errors were recognized where blurring was present in the image. Therefore, contours of individual fruits could not be precisely approximated by the algorithm.
The estimation of the fruit volumes and weights from the UAV image showed reliable correlations to the observed data. This confirms the positive results for single fruit detection and shape estimation based on the implemented algorithm. Uncertainties in prediction could be attributed to errors in image mosaicking and the occlusions from leaves and stems. The second issue has been already discussed in similar studies [12,13]. A further limitation in the prediction is given if pumpkins are growing vertically in the field because then the height of the fruit cannot be estimated from the top view image. However, this could only be observed for a small number of the pumpkins harvested.

4.2. Agronomic Consequences

Compared to a field survey from Hokkaido cultivation of the same cultivar in Germany [29], the average yield of 1.51 kg m−2 with an average number of 1.37 pumpkins per m2 was lower (2 fruits m−2, 1.91 kg m−2). Even though the plant density of 1 seedling m−2 was the same, these differences can be explained by higher fertilization (∆ N 45 kg) and drip irrigation versus rainfed water supply in our experiment. Besides lower yield, we observed a more equal fruit distribution among the weight classes compared to Reinisch and Sauer [29]. The share of marketable fruits from our detection was a little lower than from trials in North-Eastern Germany with the same cultivar, where around 80% of the harvested pumpkins reached a weight >500 g but also with a higher fertilization level (∆ N 65 kg) [38]. From this trial, a slightly lower average fruit weight (950 g compared to 990 g) was reported, which indicated a more homogenous size distribution. The relatively high share of undersize fruits is typical for the observed cultivar Uchiki Kuri and can be seen as a genetic trade-off from the production goal of small to medium pumpkins [29,39] but may be compensated by better N supply [29].
Additionally, the lower weight estimations compared to other studies can be explained through difficulties during the mapping and classification procedure, since some fruits were partially covered by leaves or stems, which led to slight underestimations, specifically for the higher share of smaller pumpkins (Figure 9 and Figure 10).
Interestingly, investigations on optimal stand densities revealed higher yield tendencies towards less plants per m2, which was not reflected in our within-field variation of the yield distribution [38]. From Figure 10, no clear pattern showed advantages from more space at the borders of the two patches. The only clear gradient was towards the south-eastern, shaded part of the field (Figure 9 and Figure 10), where less and smaller pumpkins were classified. Since radiation is the main driver for photosynthesis and, consequently, dry matter accumulation in fruits, this explains very plausibly the reduced pumpkin development close to the shading trees at that edge of the field [40]. Other resource use competition, specifically for water, is of minor importance due to lower temperatures, thus resulting less evapotranspiration from the shading trees. Hence, the producer can implement this knowledge for future pumpkin cultivation and increase seedling density in this part of the field in the next rotation.

4.3. Future Perspectives

To optimize the proposed methodology, future studies should aim to detect and reconstruct fruit shape, when they are hidden by leaves or stems. The use of multispectral and thermal sensors could be useful for this purpose, as differences in reflectance between plant components can be assumed. In addition, the methodology should be tested under further field and flight conditions (e.g., change of flight altitude, seasonality, slope of field).
Further agronomic applications from such UAV imagery processing might be in-field differentiated sustainability evaluations through deriving, e.g., radiation use efficiency or water use efficiency. In general, the described methodology is transferable to other fruits or vegetables with above ground harvest products to solve similar logistical challenges as described for Hokkaido pumpkins.

5. Conclusions

In this paper, we proposed a method to detect Hokkaido fruits and estimate their volumes and weights for yield prediction using high resolution UAV imagery. The results showed that most of the fruits could be satisfactorily identified. Misclassifications and errors in fruit geometry could be successfully reduced based on the processing steps presented. In addition, different lighting conditions, as well as shadows, in the image data could be compensated by the proposed methodology.
The volumes and weights of the pumpkins could be derived from the image data with high precision, allowing more targeted pre-harvest commercialization strategies for farmers. Since most food retailers request homogeneous lots within prescribed size or weight classes, producers can improve their sales negotiations with better knowledge of classified sales volume.

Author Contributions

Conceptualization, T.J.; methodology, L.W. and T.J.; validation, T.J. and L.W.; formal analysis, L.W. and T.J.; resources, M.K.; writing—original draft preparation, L.W., I.K., and T.J.; writing—review and editing, all; visualization, L.W. and I.K.; supervision, D.T. and T.J.; All authors have read and agreed to the published version of the manuscript.


This research received no external funding.

Data Availability Statement

Data is contained within the article


We thank three anonymous reviewers for their valuable comments.

Conflicts of Interest

The authors declare no conflict of interest.


  1. FAOSTAT. Food and Agriculture Organization of the United Nations. 2008. Available online: (accessed on 17 October 2020).
  2. Salehi, B.; Sharifi-Rad, J.; Capanoglu, E.; Adrar, N.; Catalkaya, G.; Shaheen, S.; Jaffer, M.; Giri, L.; Suyal, R.; Jugran, A.K.; et al. Cucurbita Plants: From Farm to Industry. Appl. Sci. 2019, 9, 3387. [Google Scholar] [CrossRef] [Green Version]
  3. Paris, H.S. Genetic Resources of Pumpkins and Squash, Cucurbita spp. Genet. Genom. Pineapple 2016, 111–154. [Google Scholar]
  4. Ferriol, M.; Picó, B. Pumpkin and Winter Squash. In Handbook of Plantbreeding—Vegetables I; Prohens, J., Nuez, F., Eds.; Springer: New York, NY, USA, 2008; pp. 317–349. ISBN 978-0-387-72291-7. [Google Scholar]
  5. García-Martínez, H.; Flores-Magdaleno, H.; Khalil-Gardezi, A.; Ascencio-Hernández, R.; Tijerina-Chávez, L.; Vázquez-Peña, M.A.; Mancilla-Villa, O.R. Digital Count of Corn Plants Using Images Taken by Unmanned Aerial Vehicles and Cross Correlation of Templates. Agronomy 2020, 10, 469. [Google Scholar] [CrossRef] [Green Version]
  6. Gnädinger, F.; Schmidhalter, U. Digital Counts of Maize Plants by Unmanned Aerial Vehicles (UAVs). Remote Sens. 2017, 9, 544. [Google Scholar] [CrossRef] [Green Version]
  7. Reinecke, M.; Prinsloo, T. The influence of drone monitoring on crop health and harvest size. In Proceedings of the 2017 1st International Conference on Next Generation Computing Applications (NextComp), Mauritius, Mauritius, 19–21 July 2017; Institute of Electrical and Electronics Engineers (IEEE): Piscataway, NJ, USA, 2017; pp. 5–10. [Google Scholar]
  8. Dehkordi, R.H.; Burgeon, V.; Fouché, J.; Gomez, E.P.; Cornélis, J.-T.; Nguyen, F.; Denis, A.; Meersmans, J. Using UAV Collected RGB and Multispectral Images to Evaluate Winter Wheat Performance Across a Site Characterized by Century-Old Biochar Patches in Belgium. Remote Sens. 2020, 12, 2504. [Google Scholar] [CrossRef]
  9. Furukawa, F.; Maruyama, K.; Saito, Y.K.; Kaneko, M. Corn Height Estimation Using UAV for Yield Prediction and Crop Monitoring. In Unmanned Aerial Vehicle: Applications in Agriculture and Environment; Springer Science and Business Media LLC.: Berli/Heidelberg, Germany, 2019; pp. 51–69. [Google Scholar]
  10. Li, B.; Xu, X.; Han, J.; Zhang, L.; Bian, C.; Li-Ping, J.; Xu, X. The estimation of crop emergence in potatoes by UAV RGB imagery. Plant Methods 2019, 15, 1–13. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  11. Chen, Y.; Lee, W.-S.; Gan, H.; Peres, N.A.; Fraisse, C.W.; Zhang, Y.; He, Y. Strawberry Yield Prediction Based on a Deep Neural Network Using High-Resolution Aerial Orthoimages. Remote. Sens. 2019, 11, 1584. [Google Scholar] [CrossRef] [Green Version]
  12. Chen, S.W.; Shivakumar, S.S.; Dcunha, S.; Das, J.; Okon, E.; Qu, C.; Taylor, C.J.; Kumar, V. Counting Apples and Oranges With Deep Learning: A Data-Driven Approach. IEEE Robot. Autom. Lett. 2017, 2, 781–788. [Google Scholar] [CrossRef]
  13. Apolo-Apolo, O.; Martínez-Guanter, J.; Egea, G.; Raja, P.; Ruiz, M.P. Deep learning techniques for estimation of the yield and size of citrus fruits using a UAV. Eur. J. Agron. 2020, 115, 126030. [Google Scholar] [CrossRef]
  14. Kalantar, A.; Edan, Y.; Gur, A.; Klapp, I. A deep learning system for single and overall weight estimation of melons using unmanned aerial vehicle images. Comput. Electron. Agric. 2020, 178, 105748. [Google Scholar] [CrossRef]
  15. Kamilaris, A.; Prenafeta-Boldú, F.X. Deep learning in agriculture: A survey. Comput. Electron. Agric. 2018, 147, 70–90. [Google Scholar] [CrossRef] [Green Version]
  16. Yamamoto, K.; Guo, W.; Yoshioka, Y.; Ninomiya, S. On Plant Detection of Intact Tomato Fruits Using Image Analysis and Machine Learning Methods. Sensors 2014, 14, 12191–12206. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  17. IUSS Working Group WRB. World Reference Base for Soil Resources 2014. In International Soil Classification System for Naming Soils and Creating Legends for Soil Maps; IUSS Working Group WRB: Rome, Italy, 2014; ISBN 9789251083697. [Google Scholar]
  18. Peel, M.C.; Finlayson, B.L.; McMahon, T.A. Updated world map of the Köppen-Geiger climate classification. Hydrol. Earth Syst. Sci. 2007, 11, 1633–1644. [Google Scholar] [CrossRef] [Green Version]
  19. DWD Deutscher Wetterdienst. Available online: (accessed on 1 August 2018).
  20. Meier, U. Growth Stages of Mono-and Dicotyledonous Plants, 2nd ed.; Federal Biological Research Centre for Agriculture and Forestry: Quedlinburg, Germany, 2001. [Google Scholar]
  21. Pal, M. Random forest classifier for remote sensing classification. Int. J. Remote Sens. 2005, 26, 217–222. [Google Scholar] [CrossRef]
  22. Belgiu, M.; Drăgu, L. Random forest in remote sensing: A review of applications and future directions. ISPRS J. Photogramm. Remote Sens. 2016, 114, 24–31. [Google Scholar] [CrossRef]
  23. Lawrence, R.L.; Wood, S.D.; Sheley, R.L. Mapping invasive plants using hyperspectral imagery and Breiman Cutler classifications (randomForest). Remote Sens. Environ. 2006, 100, 356–362. [Google Scholar] [CrossRef]
  24. Breiman, L.; Friedman, J.H.; Olshen, R.A.; Stone, C.J. Classification And Regression Trees; Informa UK Limited: London, UK, 2017. [Google Scholar]
  25. Bovik, A.C. Basic Binary Image Processing. In The Essential Guide to Image Processing; Elsevier BV: Amsterdam, The Netherlands, 2009; pp. 69–96. [Google Scholar]
  26. Srisha, R.; Khan, A. Morphological Operations for Image Processing: Understanding and its Applications. NCVSComs-13 Conf. Proc. 2013, 17–19. [Google Scholar]
  27. Suzuki, S.; Abe, K. Topological structural analysis of digitized binary images by border following. Comput. Vision Graph. Image Process. 1985, 29, 396. [Google Scholar] [CrossRef]
  28. Fitz-Gibbon, A.; Fisher, R. A Buyer’s Guide to Conic Fitting. In Procedings of the British Machine Vision Conference 1995, Birmingham, UK, 11–14 September 1995; British Machine Vision Association and Society for Pattern Recognition: Manchester, UK, 1995; p. 51. [Google Scholar]
  29. Reinisch, S.; Sauer, H. Sortenvergleich Hokkaido-Kürbisse 2015. Versuche im Dtsch. Gartenbau 2015, 1–6. [Google Scholar]
  30. Weiss, M.; Jacob, F.; Duveiller, G. Remote sensing for agricultural applications: A meta-review. Remote Sens. Environ. 2020, 236, 111402. [Google Scholar] [CrossRef]
  31. Panday, U.S.; Pratihast, A.K.; Aryal, J.; Kayastha, R.B. A Review on Drone-Based Data Solutions for Cereal Crops. Drones 2020, 4, 41. [Google Scholar] [CrossRef]
  32. Tsouros, D.C.; Bibi, S.; Sarigiannidis, P.G. A review on UAV-based applications for precision agriculture. Information 2019, 10, 349. [Google Scholar] [CrossRef] [Green Version]
  33. Astor, T.; Dayananda, S.; Nidamanuri, R.R.; Nautiyal, S.; Hanumaiah, N.; Gebauer, J.; Wachendorf, M. Estimation of Vegetable Crop Parameter by Multi-temporal UAV-Borne Images. Remote Sens. 2018, 10, 805. [Google Scholar] [CrossRef] [Green Version]
  34. Banko, G. A review of assessing the accuracy of and of methods including remote sensing data in forest inventory. Int. Inst. Appl. Syst. Anal. Interim Rep. 1998, 1–36. [Google Scholar]
  35. Zawbaa, H.M.; Hazman, M.; Abbass, M.; Hassanien, A.E. Automatic fruit classification using random forest algorithm. In Proceedings of the 2014 14th International Conference on Hybrid Intelligent Systems, ‎Kuwait City, Kuwait, 14–16 December 2014; Institute of Electrical and Electronics Engineers (IEEE): Piscataway, NJ, USA, 2014; pp. 164–168. [Google Scholar]
  36. Bu, R.; Xiong, J.; Chen, S.; Zheng, Z.; Guo, W.; Yang, Z.; Lin, X. A shadow detection and removal method for fruit recognition in natural environments. Precis. Agric. 2020, 21, 782–801. [Google Scholar] [CrossRef]
  37. Yu, Y.; Zhang, K.; Liu, H.; Yang, L.; Zhang, D. Real-Time Visual Localization of the Picking Points for a Ridge-Planting Strawberry Harvesting Robot. IEEE Access 2020, 8, 116556–116568. [Google Scholar] [CrossRef]
  38. Hirthe, G.; Heinze, C. Auswirkung unterschiedlicher Standweiten auf den Ertrag von Hokkaido-Kürbis im Ökologischen Anbau. Landesforschungsan. Landwirtsch. Fisch. M-V 2006, 1–5. [Google Scholar]
  39. Loy, J.B. Breeding squash and pumpkins. In Genetics, Genomics and Breeding of Cucurbits; Wang, Y.-H., Behera, T., Kole, C., Eds.; CRC Press: Boca Raton, FL, USA, 2012; pp. 93–139. [Google Scholar]
  40. Connor, D.J.; Loomis, R.S.; Cassman, K.G. Crop Ecology: Productivity and Management in Agricultural Systems; Cambridge University Press: Cambridge, UK, 2011; ISBN 9780511974199. [Google Scholar]
Figure 1. Development of cultivation area and yield in Germany (aggregated data on pumpkin, squash and gourds). Data source: Reference [1].
Figure 1. Development of cultivation area and yield in Germany (aggregated data on pumpkin, squash and gourds). Data source: Reference [1].
Sensors 21 00118 g001
Figure 2. Climate diagram with long-term average values (LTA: 1981–2010) compared to the year 2017 for monthly precipitation and mean air temperatures. Data source: Reference [19].
Figure 2. Climate diagram with long-term average values (LTA: 1981–2010) compared to the year 2017 for monthly precipitation and mean air temperatures. Data source: Reference [19].
Sensors 21 00118 g002
Figure 3. Workflow of the proposed methodology.
Figure 3. Workflow of the proposed methodology.
Sensors 21 00118 g003
Figure 4. Illustration of the tested conditions for object splitting (left and middle), as well as the implementation for multi-fruit object separation.
Figure 4. Illustration of the tested conditions for object splitting (left and middle), as well as the implementation for multi-fruit object separation.
Sensors 21 00118 g004
Figure 5. Two-dimensional representation of the pumpkin fruits and its extension in height and diameter (left), volume calculation using shape of spheroid (middle), and empirical linear regression for weight estimation of detected fruits from volume (right).
Figure 5. Two-dimensional representation of the pumpkin fruits and its extension in height and diameter (left), volume calculation using shape of spheroid (middle), and empirical linear regression for weight estimation of detected fruits from volume (right).
Sensors 21 00118 g005
Figure 6. Orthophoto with selected classification steps during fruit detection procedure.
Figure 6. Orthophoto with selected classification steps during fruit detection procedure.
Sensors 21 00118 g006
Figure 7. Scatterplot of observed and estimated fruit volume (left) and weight (right) with 1:1-line (gray) and correlation (red).
Figure 7. Scatterplot of observed and estimated fruit volume (left) and weight (right) with 1:1-line (gray) and correlation (red).
Sensors 21 00118 g007
Figure 8. Prediction errors due to overlap by leaves (left) and insufficient image mosaicking (right).
Figure 8. Prediction errors due to overlap by leaves (left) and insufficient image mosaicking (right).
Sensors 21 00118 g008
Figure 9. Field distribution of pumpkin fruits by weight classes.
Figure 9. Field distribution of pumpkin fruits by weight classes.
Sensors 21 00118 g009
Figure 10. Spatial pattern of 9 × 9 m grid cells and frequency distributions for number of fruits (left) and weights (right).
Figure 10. Spatial pattern of 9 × 9 m grid cells and frequency distributions for number of fruits (left) and weights (right).
Sensors 21 00118 g010
Table 1. Agronomic management during the cropping season 2017. DAP: days after planting.
Table 1. Agronomic management during the cropping season 2017. DAP: days after planting.
30 March−48Tillage:
Primary soil cultivation by moldboard plough (25 cm depth)
15 May−2Fertilization:
60 kg N ha−1, 60 kg P ha−1, 60 kg K ha−1
16 May−1Tillage:
Seedbed preparation by cultivator with crumbler roller
17 May0Planting:
Cultivar Uchiki Kuri, 10.000 plants ha−1, 2 m row width
24 May7Weed regulation: once a week, 5 times
Between rows by harrow, in row manually
7 June21Fertilization:
15 kg N ha−1, 15 kg P ha−1, 15 kg K ha−1
24 September130Harvest
Table 2. Descriptive statistics of pumpkin fruits (n: sample size).
Table 2. Descriptive statistics of pumpkin fruits (n: sample size).
Height (cm)1406.519.012.42.6
Diameter (cm)1406.
Weight (g)40142.03011.01187.0776.2
Table 3. Distribution of Hokkaido fruits within the weight classes for food retailing.
Table 3. Distribution of Hokkaido fruits within the weight classes for food retailing.
Weight Class (kg)Classification DataPercentage
<0.5 *62827.5
* non-marketable → forage.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Wittstruck, L.; Kühling, I.; Trautz, D.; Kohlbrecher, M.; Jarmer, T. UAV-Based RGB Imagery for Hokkaido Pumpkin (Cucurbita max.) Detection and Yield Estimation. Sensors 2021, 21, 118.

AMA Style

Wittstruck L, Kühling I, Trautz D, Kohlbrecher M, Jarmer T. UAV-Based RGB Imagery for Hokkaido Pumpkin (Cucurbita max.) Detection and Yield Estimation. Sensors. 2021; 21(1):118.

Chicago/Turabian Style

Wittstruck, Lucas, Insa Kühling, Dieter Trautz, Maik Kohlbrecher, and Thomas Jarmer. 2021. "UAV-Based RGB Imagery for Hokkaido Pumpkin (Cucurbita max.) Detection and Yield Estimation" Sensors 21, no. 1: 118.

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop