Next Article in Journal
Epigenetic Modifications and Breeding Applications in Horticultural Plants
Next Article in Special Issue
Determining Moisture Content of Basil Using Handheld Near-Infrared Spectroscopy
Previous Article in Journal
Phytochemical Analysis and Specific Activities of Bark and Flower Extracts from Four Magnolia Plant Species
Previous Article in Special Issue
Insights from a Patent Portfolio Analysis on Sensor Technologies for Measuring Fruit Properties
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Horticulturae
Volume 10
Issue 2
10.3390/horticulturae10020142
horticulturae-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

A Machine Learning-Assisted Three-Dimensional Image Analysis for Weight Estimation of Radish

by
Yuto Kamiwaki
1 and
Shinji Fukuda
2,3,*
1
United Graduate School of Agricultural Science, Tokyo University of Agriculture and Technology, Tokyo 183-8509, Japan
2
Institute of Agriculture, Tokyo University of Agriculture and Technology, Tokyo 183-8509, Japan
3
Research Center for Agricultural Information Technology, National Agriculture and Food Research Organization, Tokyo 105-0003, Japan
*
Author to whom correspondence should be addressed.
Horticulturae 2024, 10(2), 142; https://doi.org/10.3390/horticulturae10020142
Submission received: 16 January 2024 / Revised: 26 January 2024 / Accepted: 29 January 2024 / Published: 31 January 2024
(This article belongs to the Special Issue Smart Horticulture: Latest Advances and Prospects)
Browse Figures
Versions Notes

Abstract

:
The quality of radish roots depends largely on its cultivar, production environment, and postharvest management along the supply chain. Quality monitoring of fresh products is of utmost importance during the postharvest period. The purpose of this study is to nondestructively estimate the weight of a radish using random forests based on color and shape information obtained from images, as well as volumetric information obtained by analyzing a point cloud obtained by combining multiple forms of shape information. The explanatory variables were color and shape information obtained through an image analysis of still images of radishes captured in a constructed photographic environment. The volume information was calculated from the bounding box and convex hull applied to the point cloud by combining the shape information obtained from the image analysis. We then applied random forests to relate the radish weight to the explanatory variables. The experimental results showed that the models using color, shape, or volume information all exhibited good performance with a Pearson’s correlation coefficient (COR) ≥ 0.80, suggesting the potential of nondestructive monitoring of radish weight based on color, shape, and volume information. Specifically, the model using volume information showed very high performance, with a COR of 0.95 or higher.

1. Introduction

The radish (Raphanus sativus L. var. sativus) is a major root crop and its root color, shape, texture, and nutritional value vary depending on the cultivar and the cultivation environment. In the botanical classification, it is a variant of radish but is different from the oriental radish, western small radish, or black radish. The radish is a rather new cultivar in agriculture, differs from Japanese radish in origin and morphology, and is distinguished from Western species. It is mainly consumed fresh because of its late bolting, low starch content, and lack of storability. The root can be round to oblong in shape, with a cultivar of colors such as red, white, purple, and pink, of which the most widespread cultivar has a red root [1].
The shape and size of root crops, including radishes, are influenced not only by the genotype but also by environmental conditions, especially the soil type [2]. The quality of radishes is based on color and shape and depends mainly on the cultivar; however, the final quality is determined by the production environment [2,3,4,5] and postharvest management [6]. In particular, postharvest sorting and management are important processes in the supply chain because they affect the quality of radishes during distribution. However, manual sorting of individual crops is labor-intensive. The weight of radish roots is one of the quality parameters that can be measured on a scale, but the labor cost is high when dealing with a large amount of radish roots harvested on farms or in plant factories. If a large amount of measurement is required, automatic measurement using a belt conveyor can be considered; however, this increases the equipment cost. Therefore, a nondestructive method for estimating the weight of radishes after harvest on a large scale and in a short time would be beneficial for improving the efficiency of sorting operations.
There are several methods for measuring the quality of fruits and vegetables, including methods using information regarding the color of appearance [7,8,9,10,11,12,13,14], shape [2,15], and gas emitted from vegetables and fruits [16]. The method using gas generated from vegetables and fruits as a quality indicator is not easy to perform in practice because of the effects of contamination and dilution during the gas measurement. Methods that use color information include weight estimation [11,12] based on a colorimeter as a point datum and weight estimation [7,8,13,14,15] using image analysis on a two-dimensional (2D) scale. The color information in an image contains more information in space than that in a colorimeter. Additionally, the image-based method can be used to evaluate the degree of bending of vegetables and fruits. However, crops, including radishes, have three-dimensional (3D) structures. Volume estimation has been explored in previous studies using images [17,18,19]. This is because volume is a three-dimensional piece of information that provides more data than a simple 2D image. Most plants possess symmetrical shapes, but this may not always be the case if they become deformed. Previous studies have utilized this characteristic to infer the three-dimensional structure, including the volume, from 2D images. Nevertheless, if the roots are deformed, the necessary color and structural information may not be present in 2D images. Consequently, accurate reconstruction of the three-dimensional structure requires the observation of colors and structures from 360° omnidirectional images.
Three-dimensional structures can be obtained using a three-dimensional scanner (LiDAR) and a photogrammetric approach called structure-from-motion (SfM). These methods allow for the measurement of plant structural parameters, such as leaf area, leaf inclination angle, position, height, and volume [20,21], and SfM can produce a detailed three-dimensional model of a plant with color information if a set of clear images of a plant object can be captured [20]. However, three-dimensional structural reconstruction is difficult for small objects without spherical features, such as radish roots. In addition, although three-dimensional models with color information can be obtained using the SfM method, there is a limitation in the discussion on exact color information unless images are captured in a controlled light environment.
In this study, we introduce a novel method for three-dimensional structure reconstruction using contour shapes derived from image analysis. Furthermore, we employ random forests to assess the weight of radishes, incorporating color and shape information obtained from image analysis, in addition to volumetric data obtained from the reconstructed three-dimensional structure. This technology is expected to be valuable for harvest selection and quality assessment for plant breeding. To interpret the results of the random forests analyses, the importance of the variables and response curves are used to discuss the applicability of color, shape, and volumetric information as quality indices.

2. Materials and Methods

2.1. Plant Material

In this experiment, we used radishes cultivated by the authors. The cultivation period was from 3 April 2023 to 17 May 2023 in a greenhouse in Aoba ward, Yokohama city, Kanagawa Prefecture, Japan (35.55° N, 139.55° E). The cultivars used were “Sakuranbo” and “Red chime”. Only “Sakuranbo” was subjected to either full or half irrigation. Each experimental plot consisted of three replications (nine in total). The parched soil during harvest prevented the radishes from adhering to it, so they did not require washing. In this study, we added cultivar and irrigation amounts as input variables along with other variables for the random forests.

2.2. Photographic Environment and Data Acquisition Methods

To collect data on temporal changes in the weight of the radish roots, the radishes were stored in a refrigerator and the weight of each was measured at 22:00 each day. In addition, images were captured in a fixed photographic environment. The photographic environment was a box-type apparatus with a height, width, and depth of 900, 460, and 460 mm, respectively (Figure 1). In this study, white LED bulbs (color temperature: 5000 K) were used at maximum brightness (illuminance: 5440 lx). A color chart (SCK100, Datacolor, Inc., Lawrenceville, NJ, USA) and a rotating stand (NA2006, Yuanj, Shenzhen, China) were installed in the photographic environment. Because each radish had a three-dimensional structure and uneven coloration, it was fixed to a stand on a rotation table that rotated once every 50 s and was captured every 2 s with a digital camera (DC-GX7MK3, Panasonic Corporation, Tokyo, Japan). There were instances in which the camera shutter was not activated, resulting in an average of 23.5 images being captured. In this study, each image was annotated using LabelImg [22] for a more accurate extraction of color information and contour shapes.

2.3. Color Information Collection

The color information (RGB, HSL, and HSV values) was obtained from the cropped image based on the bounding box. The HSL and HSV values were calculated from RGB values. The HSL values are those of the HSL color system, where H is hue, S is saturation, and L is lightness. The values of H and S in the HSL color system are theoretically unaffected by the amount of illuminance because the lighting conditions are summarized in L [23]. Thus, it is possible to perform a color analysis independent of uneven illuminance. There is a report on the effect of lighting conditions on the HSL color system, and the saturation of the HSL color system decreases slightly when the illuminance falls to an extreme level [23]. However, because the illuminance was fixed at 5440 lx in the photographic environment in this study, the effect can be considered extremely small. The HSV color system is similar to the HSL color system, where H is hue, S is saturation, and V is brightness. In the HSV color system, the color changes to the strongest RGB color when saturation is decreased, and in the HSL color system, the color changes between the strongest and weakest RGB colors. The HSL or HSV color system ranges were H (0–360), S (0–100), and L or V (0–100). In this study, the HSL and HSV values were calculated from each RGB value using a Python library colormap [24]. The minimum, mean, median, and maximum values in the RGB, HSL, and HSV color systems were calculated for each radish on each measurement date. All color information was used as the model input for the random forests.

2.4. Shape Information Collection

After extracting the contour information of the main root of the radish from an image, the contour shape was described using elliptic Fourier descriptors (EFD) [25]. The elliptic Fourier descriptor analysis used in this study is a typical method for shape comparison based on contours, in which a closed curve, which is the contour information, is considered a periodic function and the shape is approximated and analyzed using Fourier coefficients derived from a Fourier analysis. Elliptic Fourier descriptors have been applied for the shape analysis of the Japanese radish [2] and radish [13] in previous studies. In this study, the number of harmonics was set to five because the radish cultivar used in this study was round cultivar, and the coefficients of 17 elliptic Fourier descriptors after normalization from d0 to d4 were used as input variables for the analysis in the random forests; a0, b0, and c0 were excluded because they were the same for all shape approximations.

2.5. Three-Dimensional Structure Reconstruction from Contour Information

In this study, the three-dimensional structure was reconstructed by combining contour information obtained from images captured from multiple directions. The radish was rotated 360° and captured from multiple directions at a shutter speed of one image per two seconds. Because the speed of the rotation table was one rotation per 50 s, it was theoretically possible to recover the three-dimensional shape by rotating and combining the contour shapes obtained by image analysis in three dimensions by 14.4°. This method is faster and simpler than conventional point-cloud acquisition using SfM to reconstruct three-dimensional structures. However, it is currently difficult to obtain accurate color information. The reconstructed three-dimensional structure is illustrated in Figure 2.

2.6. Volume Information Collection

The volume information was calculated from the reconstructed point cloud by combining the contour information obtained from the images. Two methods were used in this study to obtain the volume information: The first method used a bounding box and the second method used a convex hull. The second method provided a more accurate volume estimation than the first method. In this study, the volume information obtained by these two methods was used as an input variable for the random forests.

2.7. Modeling with Random Forests

The random forests [26] classifier is an ensemble learning algorithm that combines the results of multiple parallel decision trees to perform classification and regression. First, the algorithm extracts several bootstrap samples from the dataset. Second, a decision tree model is generated for each bootstrap sample. Each decision tree model uses only a few randomly selected variables as features. Finally, multiple decision tree models are calculated and the output results are obtained using the mean value for the regression tree models and majority voting for the classification tree models. The problem with decision trees is that the deeper the tree, the more complex the structure becomes and the more prone it is to overtraining. However, the random forests algorithm has a better generalization performance than decision trees because of bagging, which alleviates the problem of overlearning. Additionally, two other important features of random forests are their ability to evaluate the importance of each input variable [27] and visualize the response curve.
In this study, 15 weight estimation models were constructed based on color, shape, and volume information. Table 1 summarizes the variables used in the 15 weight estimation models. The random forests of this study were implemented using Scikit-learn, a Python library [28], and default values were used for the parameters of the random forests, except for the random seed (i.e., random_state). The random seed was changed 50 times and the mean score was calculated. In this study, five-fold cross-validation was conducted and the reproducibility of the model was evaluated based on Pearson’s correlation coefficient (COR), Nash–Sutcliffe coefficient (NSE) [29], and root mean squared error (RMSE) between the observed weights and estimated weights of radishes.
The SHapley Additive exPlanations (SHAP) [30], Partial Dependence (PD) plots, and Individual Conditional Expectation (ICE) plots were employed to interpret the models constructed in this study. SHAP applies the Shapley value concept of the cooperative game theory to deconstruct the difference between the predictions of an instance and the average prediction of the contribution of each feature. However, SHAP cannot explain how the predictions react to changes in the feature values. Thus, to visualize the structure of the weight estimation models, the PD and ICE plots were used together. PD is an interpretation method in which only certain features are moved, whereas other features are fixed, and the predictions for each instance are averaged and visualized. ICE is an interpretation method that does not average the relationship between features and model predictions for each instance.

3. Results

3.1. Time-Series Changes in Root Color and Volume of Radish

Figure 3, Figure 4, Figure 5 and Figure 6 show the time-series changes in the root color of the radishes. Figure 3b shows that the maximum value of R decreased significantly with time and the maximum value of GB decreased with time. Figure 4a shows that the mean value of S decreased significantly with time in HSL values converted from RGB values, whereas the mean values of H and L did not decrease. Similar to the HSL values in Figure 4, the mean value of S decreased significantly with time (Figure 5a), as did the HSV values converted from the RGB values (Figure 5b). The maximum H and S values remained constant, and only the V value decreased (Figure 5b). These time-series changes in color were partially different from those observed in a previous study [13]. Figure 6 shows the time-series changes in volume, which were the result of the calculation from the point cloud. Figure 7 shows the time-series changes in mean weight. The volume decreased in the same manner as the mean weight. The volume calculated from the bounding box was larger than that calculated using the convex hull (Figure 6).

3.2. Modeling Result

The results of weight estimation in the five-fold cross-validation with the random forests showed good agreement with the changes in weight over time (Table 2, Figure 8). Table 2 shows the mean and standard deviation of the evaluation results for each dataset in the five-fold cross-validation, and Figure 8 shows the results of the five-fold cross-validation. The CORs of all the models were greater than 0.8, indicating high performance in weight estimation. Among them, the CORs of the models that used volume information were all greater than 0.95, indicating that weight estimation was possible with very high accuracy. Models that used color and shape information were more accurate than those that used color information alone (Figure 8b,f,j). Models that used color and volume information were more accurate than models that used color and shape information (Figure 8c,d,g,h,k,l). In addition, models that used color and volume information were more accurate when volumes calculated from convex hulls were used than when volumes were calculated from bounding boxes (Figure 8d,h,l). Compared with the estimations of the RGB and HSL models, the HSV model improved COR, NSE, and RMSE. Models that used shape and volume information were more accurate than those that used color and shape information (Figure 8n,o).

3.3. Model Interpretation

Among the RGB+3D_convex hull model, HSL+3D_convex hull model, HSV+3D_convex hull model, and EFD+3D_convex hull model that showed good model performance, the top 10 mean absolute SHAP values for each model are shown in Figure 9. Although the mean absolute SHAP values could be calculated for other variables, only volume information was the most important. The PD and ICE plots for the top three variables for the mean absolute SHAP values in the RGB+3D_convex hull model, HSL+3D_convex hull model, and HSV+3D_convex hull models are shown in Figure 10. As shown in Figure 10a,d,g), the weight of the radishes increased with volume. Although color-based models could be applied for weight estimation (as shown in Figure 9 and summarized in Table 2), the relationship between color and weight in predictive models based on color and 3D structures remained unclear (Figure 10b,c,e,f,h,i).

4. Discussion

As the radishes in this study were grown by the authors, compared with commercial products, the quality was not consistent because the radish roots were not selected. The accuracy of the model that used only color information in this study was lower than that used in a previous study [13]. However, the mean RGB values were similar to those obtained in a previous study [13] (Figure 3a). This suggests that the minimum, maximum, or median RGB values may have been less uniform for the radishes tested in this study than for the commercial products (Figure 8a,e,i). In contrast, the model that used color and shape information was more accurate than that in the previous study [13] (Figure 8b,f,j). In a previous study, only one cultivar purchased at a supermarket was tested [13], but in this study, two cultivars (two patterns of irrigation conditions for one target cultivar) were tested and data were collected. Therefore, the accuracy of the weight estimation was higher than that in the previous study [13] because the model was able to capture changes in shape depending on the cultivar and irrigation conditions. This suggests the possibility of a more accurate crop quality estimation by building an estimation model that includes environmental conditions, as in a previous study of mangoes [5].
In addition, the volume and weight of the “Sakuranbo” cultivar were halved when the amount of irrigation was halved (Figure 6 and Figure 7). Previous studies reported that the quality of major root crops such as radishes was affected by the production environment, including irrigation. These results suggest that controlling the amount of irrigation is important not only in 3D reconstruction using shape information and modeling, such as weight estimation, but also in radish cultivation. Volume was also an important variable in this study. Volume is as important as quality information for crops, representing the size of an object. In general, the larger the volume, the heavier the weight [31,32,33,34]. Therefore, volume information was considered the most important variable in this study. In particular, this study compared the volume obtained from the bounding box with that obtained from the convex hull. The method of calculating the volume by applying the convex hull to the point cloud of the radishes provided more accurate volume information (Figure 6). The three-dimensional structural reconstruction and weight estimation methods proposed in this study can be applied to various cultivars of radishes and even to other crops, as in previous studies [31,32,33,34]. In particular, focusing on radishes suggests the possibility of constructing a weight-estimation model that transcends cultivar. However, unlike a previous study [13], the importance of color information was reduced because of the use of volume information. In recent years, a wide cultivar of radish colors has become available in the market. Therefore, color information may be an important variable when estimating weight by using data from a cultivar of colored radishes. However, the models in this study are not likely to be directly applicable to non-red cultivars because color information is only available for red cultivars. Therefore, modeling with a larger number of cultivars is a suitable approach for future studies. Additionally, as in a previous study [13], the R value of the root color of the radishes tended to decrease with time (Figure 3a). This may be due to the effect of fungal growth on the root surface [35] or oxidation of substances [36].
The volume of the radish roots decreased with decreasing water content. A decrease in water content can result in structural deformations, such as cracks and holes, which are major losses in the postharvest quality of radishes. Considering these physiological and physical changes in radishes, detailed experiments on the internal quality of radishes are expected to expand the range of applications of quality estimation based on color, shape, and volume information.

5. Conclusions

The purpose of this study was to nondestructively and accurately estimate the weight of radishes based on color and shape information obtained from images and volume information obtained from a point cloud reconstructed by combining shape information. Color and shape information, which are explanatory variables, were obtained by an image analysis of radishes taken in a certain photographic environment. The volume information was calculated from the bounding box or convex hull applied to the point cloud by combining the shape information obtained from the image analysis. The experimental results showed good reproducibility of the model that used color, shape, or volume information, suggesting the possibility of nondestructive monitoring based on the relationship between color, shape, or volume information and the weight of the radish. In this study, the images were captured in a stable light environment. Therefore, it is necessary to consider the use of images captured outdoors under sunshine. Although the results of this study were dependent on the two cultivars, it is possible that the internal quality can be estimated by constructing models for each cultivar and cultivation environment. Future work is required to study the applicability of the proposed methods for assessing the internal components of radishes.

Author Contributions

Y.K. conceptualized this study and conducted the experiment and analysis. Y.K. drafted this paper, and S.F. finalized it. S.F. supervised this research. All authors have read and agreed to the published version of the manuscript.

Funding

Y.K. is a recipient of the support by the FLOuRISH Fellowship Program for Next Generation Researchers at the Tokyo University of Agriculture and Technology.

Data Availability Statement

Data are contained within the article.

Conflicts of Interest

The authors declare no conflicts of interest.

References

  1. Kato, K.; Sato, K.; Kanazawa, T.; Shono, H.; Kobayashi, N.; Tatsuzawa, F. Relationship between Root Colors and Anthocyanins from Radishes (Raphanus sativus L.). Hortic. Res. 2013, 12, 229–234. [Google Scholar] [CrossRef]
  2. Iwata, H.; Niikura, S.; Matsuura, S.; Takano, Y.; Ukai, Y. Evaluation of variation of root shape of Japanese radish (Raphanus sativus L.) based on image analysis using elliptic Fourier descriptors. Euphytica 1998, 102, 143–149. [Google Scholar] [CrossRef]
  3. Kang, Y.; Wan, S. Effect of soil water potential on radish (Raphanus sativus L.) growth and water use under drip irrigation. Sci. Hortic. 2005, 106, 275–292. [Google Scholar] [CrossRef]
  4. Basnet, B.; Aryal, A.; Neupane, A.; Bishal, K.C.; Rai, N.H.; Adhikari, S.; Khanal, P.; Basnet, M. Effect of integrated nutrient management on growth and yield of radish. J. Agric. Nat. Resour. 2021, 4, 167–174. [Google Scholar] [CrossRef]
  5. Fukuda, S.; Spreer, W.; Yasunaga, E.; Yuge, K.; Sardsud, V.; Müller, J. Random Forests modelling for the estimation of mango (Mangifera indica L. cv. Chok Anan) fruit yields under different irrigation regimes. Agric. Water Manag. 2013, 116, 142–150. [Google Scholar] [CrossRef]
  6. Öz, A.T.; Akyol, B. Effects of calcium chloride plus coating in modified-atmosphere packaging storage on whole-radish postharvest quality. J. Sci. Food Agric. 2020, 100, 3942–3949. [Google Scholar] [CrossRef] [PubMed]
  7. Utai, K.; Nagle, M.; Hämmerle, S.; Spreer, W.; Mahayothee, B.; Müller, J. Mass estimation of mango fruits (Mangifera indica L., cv. ‘Nam Dokmai’) by linking image processing and artificial neural network. Eng. Agric. Environ. Food 2019, 12, 103–110. [Google Scholar] [CrossRef]
  8. Spreer, W.; Müller, J. Estimating the mass of mango fruit (Mangifera indica, cv. Chok Anan) from its geometric dimensions by optical measurement. Comput. Electron. Agric. 2021, 75, 125–131. [Google Scholar] [CrossRef]
  9. Fahad, L.G.; Tahir, S.F.; Rasheed, U.; Saqib, H.; Hassan, M.; Alquhayz, H. Fruits and Vegetables Freshness Categorization Using Deep Learning. Comput. Mater. Contin. 2022, 71, 5083–5098. [Google Scholar] [CrossRef]
  10. Moon, E.J.; Kim, Y.; Xu, Y.; Na, Y.; Giaccia, A.J.; Lee, J.H. Evaluation of Salmon, Tuna, and Beef Freshness Using a Portable Spectrometer. Sensors 2020, 20, 4299. [Google Scholar] [CrossRef] [PubMed]
  11. Fukuda, S.; Yasunaga, E.; Nagle, M.; Yuge, K.; Sardsud, V.; Spreer, W.; Müller, J. Modelling the relationship between peel colour and the quality of fresh mango fruit using Random Forests. J. Food Eng. 2014, 131, 7–17. [Google Scholar] [CrossRef]
  12. Caruso, G.; Palai, G.; Marra, F.P.; Caruso, T. High-Resolution UAV Imagery for Field Olive (Olea europaea L.) Phenotyping. Horticulturae 2021, 7, 258. [Google Scholar] [CrossRef]
  13. Kamiwaki, Y.; Fukuda, S. Modeling the Relationship between Root Color, Root Shape, and Weight of Radish using Machine Learning. Jxiv Preprint. [CrossRef]
  14. Victorino, G.; Poblete-Echeverría, C.; Lopes, C.M. A Multicultivar Approach for Grape Bunch Weight Estimation Using Image Analysis. Horticulturae 2022, 8, 233. [Google Scholar] [CrossRef]
  15. Amaral, M.H.; Walsh, K.B. In-Orchard Sizing of Mango Fruit: 2. Forward Estimation of Size at Harvest. Horticulturae 2023, 9, 54. [Google Scholar] [CrossRef]
  16. Wang, X.; Feng, H.; Chen, T.; Zhao, S.; Zhang, J.; Zhang, X. Gas sensor technologies and mathematical modelling for quality sensing in fruit and vegetable cold chains: A review. Trends Food Sci. Technol. 2021, 110, 483–492. [Google Scholar] [CrossRef]
  17. Chopin, J.; Laga, H.; Miklavcic, S.J. A new method for accurate, high-throughput volume estimation from three 2D projective images. Int. J. Food Prop. 2017, 20, 2344–2357. [Google Scholar] [CrossRef]
  18. Nyalala, I.; Okinda, C.; Chao, Q.; Mecha, P.; Korohou, T.; Yi, Z.; Nyalala, S.; Jiayu, Z.; Chao, L.; Kunjie, C. Weight and volume estimation of single and occluded tomatoes using machine vision. Int. J. Food Prop. 2021, 24, 818–832. [Google Scholar] [CrossRef]
  19. Huynh, T.; Tran, L.; Dao, S. Real-Time Size and Mass Estimation of Slender Axi-Symmetric Fruit/Vegetable Using a Single Top View Image. Sensors 2020, 20, 5406. [Google Scholar] [CrossRef]
  20. Itakura, K.; Hosoi, F. Automatic Leaf Segmentation for Estimating Leaf Area and Leaf Inclination Angle in 3D Plant Images. Sensors 2018, 18, 3576. [Google Scholar] [CrossRef]
  21. Itakura, K.; Kamakura, I.; Hosoi, F. Three-Dimensional Monitoring of Plant Structural Parameters and Chlorophyll Distribution. Sensors 2019, 19, 413. [Google Scholar] [CrossRef] [PubMed]
  22. Tzutalin, D. LabelImg. Available online: https://github.com/tzutalin/labelImg (accessed on 24 January 2024).
  23. Motonaga, Y.; Kameoka, T.; Hashimoto, A. Constructing Color Image Processing System for Managing the Surface Color of Agricultural Products. J. Jpn. Soc. Agric. Mach. 1997, 59, 13–22. [Google Scholar] [CrossRef]
  24. Cokelaer, T. Colormap. Available online: https://github.com/cokelaer/colormap (accessed on 24 January 2024).
  25. Kuhl, F.P.; Giardina, C.R. Elliptic Fourier features of a closed contour. Comput. Graph. Image Process. 1982, 18, 236–258. [Google Scholar] [CrossRef]
  26. Breiman, L. Random forests. Mach. Learn. 2001, 45, 5–32. [Google Scholar] [CrossRef]
  27. Cutler, D.R.; Edwards, T.C., Jr.; Beard, K.H.; Cutler, A.; Hess, K.T.; Gibson, J.; Lawler, J.J. Random forests for classification in ecology. Ecology 2007, 88, 2783–2792. [Google Scholar] [CrossRef] [PubMed]
  28. Pedregosa, F.; Varoquaux, G.; Gramfort, A.; Michel, V.; Thirion, B.; Grisel, O.; Blondel, M.; Prettenhofer, P.; Weiss, R.; Dubourg, V.; et al. Scikit-learn: Machine Learning in Python. J. Mach. Learn. Res. 2021, 12, 2825–2830. [Google Scholar] [CrossRef]
  29. Nash, J.E.; Sutcliffe, J.V. River flow forecasting through conceptual models part I—A discussion of principles. J. Hydrol. 1970, 10, 282–290. [Google Scholar] [CrossRef]
  30. Lundberg, S.M.; Lee, S.I. A unified approach to interpreting model predictions. In Proceedings of the 31st International Con-ference on Neural Information Processing Systems (NIPS’17), New York, NY, USA, 4–9 December 2017; pp. 4768–4777. Available online: https://dl.acm.org/doi/10.5555/3295222.3295230 (accessed on 27 December 2023).
  31. Hahn, F.; Sanchez, S. Carrot Volume Evaluation using Imaging Algorithms. J. Agric. Eng. Res. 2000, 75, 243–249. [Google Scholar] [CrossRef]
  32. Nyalala, I.; Okinda, C.; Nyalala, L.; Makange, N.; Chao, Q.; Chao, L.; Yousaf, K.; Chen, K. Tomato volume and mass estimation using computer vision and machine learning algorithms: Cherry tomato model. J. Food Eng. 2019, 263, 288–298. [Google Scholar] [CrossRef]
  33. Jadhav, T.; Singh, K.; Abhyankar, A. Volumetric estimation using 3D reconstruction method for grading of fruits. Multimedia Tools Appl. 2019, 78, 1613–1634. [Google Scholar] [CrossRef]
  34. Omid, M.; Khojastehnazhand, M.; Tabatabaeefar, A. Estimating volume and mass of citrus fruits by image processing technique. J. Food Eng. 2010, 100, 315–321. [Google Scholar] [CrossRef]
  35. Gálvez, L.; Palmero, D. Incidence and Etiology of Postharvest Fungal Diseases Associated with Bulb Rot in Garlic (Alllium sativum) in Spain. Foods 2021, 10, 1063. [Google Scholar] [CrossRef] [PubMed]
  36. Cömert, E.D.; Mogol, B.A.; Gökmen, V. Relationship between color and antioxidant capacity of fruits and vegetables. Curr. Res. Food Sci. 2020, 2, 1–10. [Google Scholar] [CrossRef] [PubMed]
Horticulturae 10 00142 g001
Figure 1. Schematic diagram of the experimental device.
Figure 1. Schematic diagram of the experimental device.
Horticulturae 10 00142 g001
Horticulturae 10 00142 g002
Figure 2. Example of 3D structure reconstruction results: (a) front view, (b) downward view, (c) lateral view.
Figure 2. Example of 3D structure reconstruction results: (a) front view, (b) downward view, (c) lateral view.
Horticulturae 10 00142 g002
Horticulturae 10 00142 g003
Figure 3. Time-series changes in RGB values: (a) mean RGB values, (b) maximum RGB values. Mean ± standard error is shown ( n = 15 ).
Figure 3. Time-series changes in RGB values: (a) mean RGB values, (b) maximum RGB values. Mean ± standard error is shown ( n = 15 ).
Horticulturae 10 00142 g003
Horticulturae 10 00142 g004
Figure 4. Time-series changes in HSL values: (a) mean HSL values, (b) maximum HSL values. Mean ± standard error is shown ( n = 15 ).
Figure 4. Time-series changes in HSL values: (a) mean HSL values, (b) maximum HSL values. Mean ± standard error is shown ( n = 15 ).
Horticulturae 10 00142 g004
Horticulturae 10 00142 g005
Figure 5. Time-series changes in HSV values: (a) mean HSV values, (b) maximum HSV values. Mean ± standard error is shown ( n = 15 ).
Figure 5. Time-series changes in HSV values: (a) mean HSV values, (b) maximum HSV values. Mean ± standard error is shown ( n = 15 ).
Horticulturae 10 00142 g005
Horticulturae 10 00142 g006
Figure 6. Time-series changes in volume: (a) volume from bbox, (b) volume from convex hull. Mean ± standard error is shown ( n = 15 ).
Figure 6. Time-series changes in volume: (a) volume from bbox, (b) volume from convex hull. Mean ± standard error is shown ( n = 15 ).
Horticulturae 10 00142 g006
Horticulturae 10 00142 g007
Figure 7. Time-series changes in mean weight. Mean ± standard error is shown ( n = 15 ).
Figure 7. Time-series changes in mean weight. Mean ± standard error is shown ( n = 15 ).
Horticulturae 10 00142 g007
Horticulturae 10 00142 g008
Figure 8. Results of weight estimation with random forests using color information, shape information, and volume information: (a) RGB model, (b) RGB+EFD model, (c) RGB+3D_bbox model, (d) RGB+3D_convex hull model, (e) HSL model, (f) HSL+EFD model, (g) HSL+3D_bbox model, (h) HSL+3D_convex hull model, (i) HSV model, (j) HSV+EFD model, (k) HSV+3D_bbox model, (l) HSV+3D_convex hull model, (m) EFD model, (n) EFD+3D_bbox model, and (o) EFD+3D_convex hull model.
Figure 8. Results of weight estimation with random forests using color information, shape information, and volume information: (a) RGB model, (b) RGB+EFD model, (c) RGB+3D_bbox model, (d) RGB+3D_convex hull model, (e) HSL model, (f) HSL+EFD model, (g) HSL+3D_bbox model, (h) HSL+3D_convex hull model, (i) HSV model, (j) HSV+EFD model, (k) HSV+3D_bbox model, (l) HSV+3D_convex hull model, (m) EFD model, (n) EFD+3D_bbox model, and (o) EFD+3D_convex hull model.
Horticulturae 10 00142 g008
Horticulturae 10 00142 g009
Figure 9. The top 10 mean absolute SHAP values of the best model in the weight estimation with random forests: (a) RGB+3D_convex hull model, (b) HSL+3D_convex hull model, (c) HSV+3D_convex hull model, and (d) EFD+3D_convex hull model.
Figure 9. The top 10 mean absolute SHAP values of the best model in the weight estimation with random forests: (a) RGB+3D_convex hull model, (b) HSL+3D_convex hull model, (c) HSV+3D_convex hull model, and (d) EFD+3D_convex hull model.
Horticulturae 10 00142 g009
Horticulturae 10 00142 g010
Figure 10. Partial dependence and individual conditional expectations in the best model for weight estimation with random forests: (a) volume from convex hull (RGB+3D_convexhull model), (b) G_mean (RGB+3D_convex hull model), (c) G_median (RGB+3D_convex hull model), (d) volume from convex hull (HSL+3D_convex hull model), (e) H_mean (HSL+3D_convex hull model), (f) S_mean (HSL+3D_convex hull model), (g) volume from convex hull (HSV+3D_convex hull model), (h) S_mean (HSV+3D_convex hull model), and (i) H_mean (HSV+3D_convex hull model).
Figure 10. Partial dependence and individual conditional expectations in the best model for weight estimation with random forests: (a) volume from convex hull (RGB+3D_convexhull model), (b) G_mean (RGB+3D_convex hull model), (c) G_median (RGB+3D_convex hull model), (d) volume from convex hull (HSL+3D_convex hull model), (e) H_mean (HSL+3D_convex hull model), (f) S_mean (HSL+3D_convex hull model), (g) volume from convex hull (HSV+3D_convex hull model), (h) S_mean (HSV+3D_convex hull model), and (i) H_mean (HSV+3D_convex hull model).
Horticulturae 10 00142 g010
Table 1. List of the constructed models and explanatory variables for weight estimation.
Table 1. List of the constructed models and explanatory variables for weight estimation.
ModelsExplanatory Variables
CultivarIrrigationRGBHSLHSVEFDVolume_bboxVolume_convex hull
RGB***
RGB+EFD*** *
RGB+3D_bbox*** *
RGB+3D_convex hull*** *
HSL** *
HSL+EFD** * *
HSL+3D_bbox** * *
HSL+3D_convex hull** * *
HSV** *
HSV+EFD** **
HSV+3D_bbox** * *
HSV+3D_convex hull** * *
EFD** *
EFD+3D_bbox** **
EFD+3D_convex hull** * *
Table 2. Model performance of random forests with respect to Pearson’s correlation coefficient (COR), Nash–Sutcliffe Efficiency (NSE), and root mean squared error (RMSE), for which mean ± standard deviation values are presented ( n = 250 ). Models were built using different sets of color information (i.e., RGB, HSL, and HSV), shape information (EFD), and volume information.
Table 2. Model performance of random forests with respect to Pearson’s correlation coefficient (COR), Nash–Sutcliffe Efficiency (NSE), and root mean squared error (RMSE), for which mean ± standard deviation values are presented ( n = 250 ). Models were built using different sets of color information (i.e., RGB, HSL, and HSV), shape information (EFD), and volume information.
Model NameCORNSERMSE
RGB0.829 ± 0.05800.648 ± 0.1261.62 ± 0.321
RGB+EFD0.905 ± 0.05820.793 ± 0.1091.21 ± 0.261
RGB+3D_bbox0.961 ± 0.02740.915 ± 0.05570.765 ± 0.217
RGB+3D_convex hull0.980 ± 0.01090.955 ± 0.02340.571 ± 0.147
HSL0.856 ± 0.05820.701 ± 0.1131.49 ± 0.325
HSL+EFD0.908 ± 0.05740.798 ± 0.1061.20 ± 0.268
HSL+3D_bbox0.970 ± 0.02180.935 ± 0.04490.671 ± 0.201
HSL+3D_convex hull0.984 ± 0.008670.963 ± 0.01960.519 ± 0.140
HSV0.870 ± 0.05710.726 ± 0.1121.42 ± 0.318
HSV+EFD0.912 ± 0.05610.806 ± 0.1031.18 ± 0.262
HSV+3D_bbox0.972 ± 0.02130.939 ± 0.04370.645 ± 0.198
HSV+3D_convex hull0.984 ± 0.008460.964 ± 0.01880.509 ± 0.137
EFD0.887 ± 0.06790.761 ± 0.1381.30 ± 0.297
EFD+3D_bbox0.963 ± 0.02350.918 ± 0.04920.760 ± 0.218
EFD+3D_convex hull0.980 ± 0.009660.953 ± 0.02330.585 ± 0.159
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Kamiwaki, Y.; Fukuda, S. A Machine Learning-Assisted Three-Dimensional Image Analysis for Weight Estimation of Radish. Horticulturae 2024, 10, 142. https://doi.org/10.3390/horticulturae10020142

AMA Style

Kamiwaki Y, Fukuda S. A Machine Learning-Assisted Three-Dimensional Image Analysis for Weight Estimation of Radish. Horticulturae. 2024; 10(2):142. https://doi.org/10.3390/horticulturae10020142

Chicago/Turabian Style

Kamiwaki, Yuto, and Shinji Fukuda. 2024. "A Machine Learning-Assisted Three-Dimensional Image Analysis for Weight Estimation of Radish" Horticulturae 10, no. 2: 142. https://doi.org/10.3390/horticulturae10020142

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop